Advogato blog for adulau Advogato blog for adulau en-us mod_virgule Tue, 24 Jan 2017 23:19:13 GMT Mon, 1 Apr 2013 22:04:22 GMT 2013-04-01 Information Visualization Is Just A Starting Point <p> <a href="" ><img src="" width="320" height="228" alt="Keywords in Common Vulnerabilities and Exposures"/></a> <a href="" ><img src="" width="320" height="194" alt="Quantit&#xE9; des d&#xE9;chets m&#xE9;nagers collect&#xE9;s sur la Province de Luxembourg - ann&#xE9;e 2012"/></a> </p><h2 id="Heading1">Information Visualization Is Just A Starting Point</h2><p>Information visualization is not an end but just a step to improve our understanding of data. Following a small discussion in the train about the visualisation of open data, I did a small experiment to analyse the statistics about the waste collection in my region. The result of <a href="" >this experiment</a> is available along with <a href="" >some random notes</a>. But the main question came from someone else looking at the visualization and basically told me: <em>"I don't get it"</em>. He is right, the experimentation is just there to trigger more analysis (and sometime more visualization) with the objective to improve our understanding. Initially, the source of data is usually not analysed and sitting there waiting to be understood. Coming back to the data about waste collection, the initial discussion about the understanding or interpretation wouldn't be triggered if the first step of visualization is not done.</p><p>So in that scope, I tried a similar approach with a dataset I built from my <a href="" >cve-search tool</a>. My idea was to see the terms used all the description of the Keywords in Common Vulnerabilities and Exposures (CVE). I did a <a href="" >first CVE terms visualization experiment</a> and then I twitted about it. Then, this was triggering various explanations like why there is a predominance of some terms as commented by Steve Christey.</p><p> <img src=""/></p><p>It clearly showed that is an iterative process especially to better understand the data. It's also an interactive process in order to improve the visualization and the data source. Following the good advise from Joshua J. Drake, I added a <a href="" >lemmatizer</a> to keep only the root of each term and also exclude the standard English stopwords. With the visualization, we saw from some occurrences (e.g. unknown or unspecified) that the CVEs are based on incomplete information.</p><p> <img src=""/></p><p>I'm quite sure that is not finished and just the beginning of more work and experiments in visualization. I read <a href=";collection=-1&amp;deepsearch=infovis" >various books about information visualization</a> but the result is often very static and you don't really see their iterative process to reach their visualization goals. Sometime, you just see a result without the process and the tools used to make the visualization happens.</p><p>At least with free software like <a href="" >D3.js</a>, we have now a set of tools to understand how the visualization was built and maybe improve/discuss those visualizations. At least, if you want to play or improve the <a href="" >visualization of terms used for software vulnerabilities</a> description, let me know.</p><p> <em>You want an open mind, but not an empty head. Just because something is a new or fashionable alternative, doesn’t mean we need to get stupid when judging it. Edward Tufte.</em> </p> Sat, 23 Feb 2013 21:04:19 GMT 2013-02-23 Vulnerability Management Is Just An Approximation <p> <a href="" ><img src="" width="320" height="213" alt="Everybody needs a hacker"/></a> </p><h2 id="Heading1">Software Vulnerability Management Is Just A Huge Approximation</h2><p>Approximation is a representation of something that is not exact. To be extremely exact vulnerability management is not even a mathematical approximation like we know it for Pi value. But from where this utterly huge approximation is coming from? The first origin is the inner definition of "vulnerability management". If you look at various definitions like the one from Wikipedia or some information security standards, you have something like "it's a process identifying → classifying → remediation → mitigation of software vulnerabilities". Many information security vendors might told that is an easy problem but you can ask yourself if this is an easy problem why so many organizations are still compromised with software vulnerabilities.</p><p>In my pragmatic eyes, it's very broad, so broad that a first reaction is to split the problems into parts that you can solve. If we just look at the initial step to identify software vulnerabilities.</p><p>To solve this problem, the first part is to discover, know and understand the software vulnerabilities. Everyone is discovering vulnerabilities everyday (just look at how many bug reports are going into the Linux Kernel bug tracking software) and very often when you report a bug, you don't even know if this is a software vulnerability. The worst part is that an organization (or an individual) doesn't exactly know what software they are running. If someone is telling you that they have a "software vulnerability management" software that is able to detect all the software running on a system, it's a lie. If such software would exist, you would have the perfect software that would be able to solve the virus detection issue while solving the Turing's halting problem. Just look at a simple software appliance and the set of software required to run the appliance.</p><p>Discovering vulnerabilities might be easy but it's difficult to be exhaustive. Even if a vulnerability is found, there is a market to limit their publications (like zero-day vulnerability market). For a named software, there is might be a large set of unknown vulnerabilities (I'm tempted to talk about Java but I think every software might fall into that category). Does this mean that you should give up? I don't think so. You must work on your vulnerability management but don't trust blindly solutions that claim to solve such issue.</p><p>Finally, my post is not a bashing post as it was an opportunity for me to talk about a side project I'm working to ease collecting and classifying Common Vulnerabilities and Exposures (CVE). The project is called <a href="" >cve-search</a> and it's not a complete vulnerability management just a small tool to solve partially the identification and the classification part.</p><p> <em>“When he time comes to leave, just walk away quietly and don't make any fuss.”– Banksy</em> </p><p><a href="" >infosec</a> <a href="" >security</a> <a href="" >vulnerability</a></p> Sun, 25 Dec 2011 16:11:19 GMT 2011-12-25 Against SOPA or How To Do Soap <div> <h2>Contents</h2> <li> <a href="#Heading1" >I'm against SOPA... So I'll explain how to make soap with olive oil</a> <ol><li> <a href="#Heading2" >Safety Disclaimer</a> </li> <li> <a href="#Heading3" >Background of the chemical process</a> </li> <li> <a href="#Heading4" >Ingredients</a> </li> <li> <a href="#Heading5" >Process</a> </li> </ol></li> </div><h1 id="Heading1">I'm against SOPA... So I'll explain how to make soap with olive oil</h1><p>One more time, some lobbyists try to regulate the Internet with some of the stupidest laws or rules. <a href="" >SOPA (in US)</a> is again one of this tentative to break down the freedom of citizen worldwide to preserve some archaic business model. As I have a preference for concrete action leading to a direct social improvement, I'll explain how to do soap (it's better than SOPA and more useful, please note the clever inversion of the letters). My soap recipe is released under the <a href="" >public domain dedication license (CC0)</a>.</p><p> <a href="" ><img src="" width="240" height="160" alt="Stop SOPA make SOAP"/></a><a href="" ><img src="" width="240" height="160" alt="Stop SOPA make SOAP"/></a> </p><h2 id="Heading2">Safety Disclaimer</h2><p>Doing soap is a chemical process that requires your full operating brain. Especially that you'll use sodium hydroxide that is a corrosive substance. So respect the proportions, the process and read the whole process multiple times before doing it. Wearing protective gloves and goggles is highly recommended. Avoid to use kitchen instruments in aluminum as it will be attacked by the sodium hydroxide.</p><h2 id="Heading3">Background of the chemical process</h2><p>Doing soap is one of the first chemical process discovered by the humanity. The process is called <a href="" >saponification</a> that is done by using a base to hydrolyze the triglycerides contained in the fats (organic or animal). This process generates a fatty acid salt along with the glycerol (the greasy touch of the soap). Each fat has a specific value for its saponification. The saponification value (usually called SAP in saponification tables) is expressed by the required volume of base (usually sodium hydroxide) to saponify 1 gram of fat. The saponification value is reduced to keep the resulting soap a bit fat (what is called the "excess fat"). I find it even convenient to keep a "safety" bound to ensure that the hydrolyze is complete and used the whole sodium hydroxide.</p><p>So that's the basis if you want to build your own soap, there are other rules to consider but for this recipe this is enough. In my case, I use olive oil as a fat. Easy to find and I have a preference for organic olive oil (to ensure that the oil producer is taking care of its environment). But you can use non-organic olive oil too (it's usually cheaper).</p><h2 id="Heading4">Ingredients</h2><ul><li>1000 grams of olive oil</li> <li>124 grams of pure <a href="" >sodium hydroxide</a> / NaOH (as the olive oil has a SAP factor of 0.134 and we want 7% of over fat → run <a href="" >bc</a> and type (1000*0.134)*0.930) (total weight of fat *SAP factor for the fat)*(0.900&lt;-&gt;0.960))</li> <li>350 grams of tap water (usually between 31% and 35% of the total fat. In this recipe ~ 1000*0.350)</li> </ul><h2 id="Heading5">Process</h2><ul><li>Put your protective gloves and goggles</li> <li>Prepare the sodium hydroxide by putting the sodium hydroxide in water (<strong>!put the sodium hydroxide in water not the reverse!</strong>).</li> <li>and monitor the temperature of the prepared sodium hydroxide to reach around 46-47 Celcius degree (it will start at 80 Celcius degree with the reaction).</li> <li>At the same time, warm the olive oil until 46-47 Celcius degree.</li> <li>When both are at the same temperature (around 46-47 Celcius degree),</li> <li>you can start to mix (using a mixer speed up the process) the warmed olive oil by incorporating the prepared sodium hydroxide. (<strong>!use a large pot to avoid projection of the prepared sodium hydroxide while mixing!</strong>).</li> <li>When you start to see that the mixture is becoming consistent (especially that you can see a trace while removing the mixer) it means that's you reach the critical point.</li> <li>When you have an homogeneous consistence, you can put the result into a plate.</li> <li>Put a plastic film into the plate touching the mixture (to avoid oxygen to be in contact with the prepared soap).</li> <li>In the next hours, you'll the "gelification process" where the soap is becoming a gel (usually starting from the center).</li> <li>After 24 hours, your soap is becoming harder. (see above picture)</li> <li>You can can remove it from the plate and cut the forms you want from your block soap.</li> <li>And the soap must dry for the next 4 weeks in a dry and clean place. (see above picture)</li> </ul><p>Tags: <a href="" >soap</a> <a href="" >sopa</a> <a href="" >freedom</a> <a href="" >chemistry</a> <a href="" >diy</a></p> Sat, 17 Dec 2011 12:11:19 GMT 2011-12-17 Certificate Revocation Reasons 2011 <p>This page is too big to send over RSS.</p> Sun, 2 Oct 2011 12:03:03 GMT 2011-10-02 Try and Vet Tshirt Crypto Challenge Hack.lu2011 The Solution <p> <a href="" ><img src="" width="500" height="333" alt="Try and Vet T-Shirt Cryptographic Contest at 2011"/></a> </p><div> <h2>Contents</h2> <li> <a href="#Heading1" >The Challenge</a> <ol><li> <a href="#Heading2" >What Did You Get During 2011?</a> </li> </ol></li> <li> <a href="#Heading3" >Solution</a> <ol><li> <a href="#Heading4" >Where Is The Key?</a> <ol><li> <a href="#Heading5" >The original text from Phrack (original.txt)</a> </li> <li> <a href="#Heading6" >The text from the 2011 t-shirt (modified.txt)</a> </li> </ol></li> </ol></li> </div><h1 id="Heading1">The Challenge</h1><h2 id="Heading2">What Did You Get During 2011?</h2><p>From <a href="" >the website</a>, you got a text message including a message stream. During the conference, you got a t-shirt.</p><p> <code> The horrible "Beer Scrunchie" subverted the 2011 conference to hide some cryptographic materials. He especially abused the t-shirt for 2011 to transmit under cover activities. We still don't know at which extend "Beer Scrunchie" abused the t-shirt. Everything is possible just like those trojan t-shirts discovered... </code> </p><pre>U2FsdGVkX19EAnHXVRgs2oajPS0zZ3+w8BlYdQbHMTI7GT9gvdgFkjtTarpNAmbz ET8PRg72U8pydsLr4IaTt5n7fFz6jxyglU1ozZwjJhKAyPAftqxYvcnud4/cOiEV 2FutxaJYCORWsvQV+hi6j8LMqn5aJd7s2nhQ9BWji/ZjMZx/wXJVdCCmNL9HuWx9 q0KV/8nTaxOOEdGwENZT8rgSSb7qy5mcIlIBfdzqYAzynj8xLxHFmptNQfZaO3X0 MAbvS324WDeB3R5p6CaIDLeH95eN8jrqdXaDhxs1SrlJrq5inssTgsEttFUhHEe8 6unUI3i4sDeVvEcajMmxvKg0qQLqEkc56GXKXVuGYc+owEsgKW8JKk8DrfgbQMPy mbaaN7h1PKjlXTIfkR9KXOMd0wy/KHEoM6FdWY1jjzB2Q9UODxgug6gNXciVpQB6 fpvlzvFkV8z8BfSMcDCo1GM6526hSYYtRF0RS3PoloSPjfvDCNVX86lMjKsx6etc Wec6u4EuJVDI52dgSr3kslwlfswez4WM+H2cszKCf0xejql/tQsra6QAcj1JhSqD C6AvtDV31IzLAhHy5Di4T1ONyk68WNU40BIsrNkb3lYFTtWtQeF5Z4DGwpcM9HKg CbLIe9oiNONgrY+kn5RfkHgUaI/PbUQgWy/U6BkunbuqTuMXwiTeR3eaRwBnGQGJ KL+w6duxhoZhCa9nrlr3I2Nx2l+bs9JIzp5h2nYIq6yhqAyQ6jE+lpAQk912FE1O 5AuOLW5bhMldPMVMlYlx6w== </pre><h1 id="Heading3">Solution</h1><p>The message on the website gave already some clues especially that:</p><ul><li>You have to look at the t-shirt for some "cryptographic materials".</li> <li>"Beer Scrunchies" is the anagram of <a href="" >"Bruce Schneier"</a> and especially that he is the author or co-author of many ciphers <a href="" ></a>.</li> </ul><p>If you decode the message encoded in Base64, you'll see that the stream of data in binary is starting in the following way : "Salted__…." That's the behaviour of the OpenSSL<a href=";id=OpenSSL" >?</a> salted encryption scheme prefixing with "Salted__" to announce that the first 8 bytes of the encrypted stream are <a href="" >reserved for the salt</a>. This gives the indication that the message has been probably encrypted with an OpenSSL<a href=";id=OpenSSL" >?</a> tool or library. If you look carefully look at the encryption schemes available in OpenSSL<a href=";id=OpenSSL" >?</a>:</p><pre>aes-128-cbc aes-128-ecb aes-192-cbc aes-192-ecb aes-256-cbc aes-256-ecb base64 bf bf-cbc bf-cfb bf-ecb bf-ofb cast cast-cbc cast5-cbc cast5-cfb cast5-ecb cast5-ofb des des-cbc des-cfb des-ecb des-ede des-ede-cbc des-ede-cfb des-ede-ofb des-ede3 des-ede3-cbc des-ede3-cfb des-ede3-ofb des-ofb des3 desx rc2 rc2-40-cbc rc2-64-cbc rc2-cbc rc2-cfb rc2-ecb rc2-ofb rc4 rc4-40 </pre><p>There are not so many algorithms written by Bruce Schneier in a default OpenSSL<a href=";id=OpenSSL" >?</a> except Blowfish (bf-*). Usually cryptographer recommends to use the "default" mode and in this case, bf is Blowfish in CBC mode. So this is highly probable…</p><h2 id="Heading4">Where Is The Key?</h2><p>As you didn't use the t-shirt until now, there is a good guess that the key is hidden somewhere. If you look carefully at the text in the back of the 2011 t-shirt, you'll see many typographic errors. The interesting part is to compare the typographic errors from the original text <a href=";id=3#article" >as published by Phrack</a>. Please note the typo in the URL (even if the URL works, doesn't mean that's the correct one ;-).</p><h3 id="Heading5">The original text from Phrack (original.txt)</h3><pre>This is our world now... the world of the electron and the switch, the beauty of the baud. We make use of a service already existing without paying for what could be dirt-cheap if it wasn't run by profiteering gluttons, and you call us criminals. We explore... and you call us criminals. We seek after knowledge... and you call us criminals. We exist without skin color, without nationality, without religious bias... and you call us criminals. You build atomic bombs, you wage wars, you murder, cheat, and lie to us and try to make us believe it's for our own good, yet we're the criminals. Yes, I am a criminal. My crime is that of curiosity. My crime is that of judging people by what they say and think, not what they look like. My crime is that of outsmarting you, something that you will never forgive me for. I am a hacker, and this is my manifesto. You may stop this individual, but you can't stop us all... after all, we're all alike. The Conscience of a Hacker, The Mentor, January 8, 1986,;id=3#article </pre><h3 id="Heading6">The text from the 2011 t-shirt (modified.txt)</h3><pre>This is our world now... the world of the electron and the swich, the beauty of the baud, We make use of a service already exeisting without paying for what could be dirt-cheep if it was'nt run by profofiteering gluttons, and you call us cricriminal. We explore... and you call us criminals. We seek after knowledge... and you call us criminals. We exist without skin colo, without nationlity, without rrligious bias... and you call us crimnals. You build atomic bombs, you wage wars, you murder, cheat, and lie to us and try to make us believe it's for our own good, yet we're the criminals. yes, I am a criminal. My crime is that of curiosity. my crime is that of judginfg people by what thy say and think, not what they look like. my crime is that of outmarting you, something that you will never forgive me for. I am a hacker, and this is my manifasto. you may stop this individul, but you can't stop us all... after all, we're all alike. The Conscience of a Hacker, The Mentor, January 8, 1986,$id=3#article </pre><p>So you can build a key from the differences but how? That's the most difficult part (as there are many different way to do it). As there is no natural way to generate a key, I decided to go for a long key that can be read easily from the original text. To build back the key from original to modified you can use word diff and use your favorite GNU tools for word diff. We just discarded the punctuation and we didn't care about the case sensitivity.</p><pre>wdiff -i -3 original.txt modified.txt | egrep -o "(\[-(.*)-\])" | sed -e "s/-//g" | sed -e "s/\[//g" | sed -e "s/\]//" | sed -e "s/\.$//g" | sed -e "s/,//g" | sed ':a;N;$!ba;s/\n//g' </pre><p>The key to decrypt the message generated from the above wdiff is the following:</p><pre>switchbaudexistingdirtcheapwasn'tprofiteeringcriminalscolornationalityreligiouscriminalsjudgingtheyoutsmartingmanifestoindividual;id=3#article </pre><p>and to decrypt the message, you'll need to use OpenSSL<a href=";id=OpenSSL" >?</a> in the following way used the guessed parameters:</p><pre>openssl enc -d -a -bf -in encrypted.txt -out decrypted.txt </pre><p>and the original decrypted message is:</p><pre>I'm Beer Scrunchie and I'm the author or co-author of various block ciphers, pseudo-random number generators and stream ciphers. In 2012, there will be two major events: the proclamation of a winner for the NIST hash function competition and probably the 2012 infosec conference . I hope that my Skein hash function will be the winner. If you are reading this text and be the first to submit to, you just won a ticket for next year. If I'm winning the NIST competition wit h my hashing function, you'll get a second free ticket... Bruce </pre><p>I got one correct answer 5 days after the conference showing that the difficulty to get back the key was bound to the uncertainty of the key generation. Next year, it's possible that we make a multi-stage t-shirt challenge for 2012… from something more easy to something very difficult.</p><p>Tags: <a href="" >crypto</a> <a href="" >infosec</a> <a href="" >ctf</a> <a href="" >conference</a> <a href="" >hacklu</a></p> Sun, 4 Sep 2011 17:02:58 GMT 2011-09-04 Information Security Is Not a Matter of Compliance <p> <a href="" ><img src="" width="240" height="160" alt="A Radio On a Piano"/></a> </p><h1 id="Heading15">Information Security Is Not a Matter Of Compliance But a Matter of Some Regular and Boring Activities</h1><p>Making conclusions from experience is not always a scientific approach but a blog is a place where to share experience. Today, I would like to share my past experience with information security and especially how much it's difficult to reach some security with the specific compliance detour proposed by the industry or even the society.</p><h2 id="Heading16">Compliance is a Different Objective Than Information Security</h2><p>Many compliance mechanisms exist in the information security to ensure on paper the security of a service, a company, a process. I won't list all of them but you might know PCI-DSS, TS 101 456, ISO/IEC 27001 and so on… Very often the core target of a company is to get the final validating document at the end of the auditing process.</p><p>Of course, many of those validation processes are requiring many strong security requirements on the procedural aspect of the information security management within the company. This is usually a great opportunity for the information security department to increase somehow their budget or their attraction. Everything is nice. But usually when the paper work is finished, the company got their golden certificate and the investment in information security is just put aside.</p><p>But concrete information security is composed of many little dirty jobs that no one wants really do. Usually in the compliance documents those tasks are underestimated (e.g. a check-box at the end of a long list) or even not mentioned (e.g. discarded during the risk assessment because they seem insignificant). Those tasks are usually a core part of information security. Not only for protecting but also to detect misuse earlier.</p><p>I summarized the tasks in three large groups (it's not an exhaustive view) but show some of the core jobs to be performed in the context of protecting information systems:</p><h3 id="Heading17">Reading and Analyzing Never Ending Log Files</h3><p>The log analysis is usually the main trigger to find a compromised system. When Clifford Stoll found that the system was compromised at LBL, it was due to a specific <a href="" >75 cents accounting issue</a>. Like the <a href="" >recent security breach at</a> discovered by an error in the log from a non-installed software (Xnest) or <a href="" >a pop up of an invalid certificate</a>, that's how infection or compromised infrastructure get discovered.</p><p>But to discover those discrepancies, you need someone at the end. The answer, here, is not a machine to read your logs (I already hear the SIEM vendors claiming this can be automatized). It's a human having some knowledge (with some doubts) to pick something unusual that can lead to the detection of something serious.</p><p><strong>The log analysis is a tedious work that needs curious and competent people.</strong> It's something difficult to describe in a compliance document. The analysis job can be boring and not really rewarded. That's why sometime you see the idea of "outsourcing" log analysis but can an outsourced analysis detect an accounting issue because he knows that some user is not working during that time shift?</p><p>IMHO, it's sometime better to invest into people and promote the act of regular logs analysis than pursue into an additional security certification without the real security activities associated.</p><h3 id="Heading18">Reducing the Attack Surface</h3><p><strong>The less software you have the better it is for its security.</strong> It sounds very obvious but that's a core concept. We pile more and more features in each software used. I never saw a control in a security standard or certification that recommends to have a policy to reduce software or remove old legacy systems. If you carefully look at "Systems Development Life Cycle", this always shows the perfect world without getting rid of old crappy code.</p><h3 id="Heading19">Maintaining the Software and Hardware</h3><p>Maintaining software and hardware could fall into the category of "reducing the attack surface" but it's another beast, often under estimated in many security compliance processes. A software is like a living organism, you have to care of it. You don't acquire a tiger and put in your garden without taking care of it. Before maintaining, you obviously need to design systems with "flaw-handling in mind" as <a href="" >Marcus J. Ranum said</a> or <a href="" >Wietse Venema</a> or <a href="" >Saltzer and Schroeder</a> in 1975 . In today's world, we are always not going in that direction so you have to maintain the software to keep out the daily security vulnerabilities.</p><p>The main issue with a classical information system is the interactions with the other systems and its environment. If you (as a security engineer) recommend to update a software in a specific infrastructure, you always hear the same song "I can't update it", "It will be done with the yearly upgrade" (usually taking 4 years), "Do you know the impact of this update on my software?" (and obviously you didn't write his software), "It's done" (while checking it's still giving the old version number), "It's not connected so we don't need to patch" (looking at the proxy logs you scare yourself by the volume of data exchanged) and … the classical "it's not managed by us" (while reading the product name in the title of the user who answers that).</p><p><strong>Yes, upgrading software (and hardware) is a dirty job, you have to bother people, chase them every days.</strong> Even in information security, upgrading software is a pain and you usually break stuff.</p><p>All those dirty jobs are part of protecting information systems, we have to do them. Security certification is distracting a lot of professionals from those core activities. <strong>I know it's arduous to do them and not rewarded, but we have to do those tasks if we want to make the field more difficult for the attackers.</strong></p><p> <em>You might ask why a picture with a radio on a piano… both can do the same "music" but are operated in a different way. Just like information security on a system or an paper are done in two different ways.</em> </p><p>Tags: <a href="" >infosec</a> <a href="" >compliance</a></p> Tue, 24 May 2011 23:48:09 GMT 2011-05-22 Ease Your Log Analysis with Ranking <p> <a href="" ><img src="" width="240" height="160" alt="Apocalypse de milieu de terrain / Mittelfeldapokalypse (Tim Ernst)"/></a> </p><h1 id="Heading1">Ease Your Log Analysis With BGP Ranking and logs-ranking</h1><p><a href="" >Raphael</a> Vinot and <a href="" >I</a> worked on a network security ranking project called <a href="" >BGP Ranking</a> to track the malicious activities per Internet Service Provider (referenced with their ASN Autonomous System Number). The project is free software and can be <a href="" >downloaded, forked or updated</a> at GitHub. As BGP Ranking recently reached a beta stage, we have now a nice set of data about the ranking of each Internet service provider in the world. Every day, we are trying to find new ways to use the dataset to improve our life and remove the boring work while doing network forensic.</p><p>A very common task when you are doing network forensic is to analyse huge stack of logs files. Sometime, you don't even know where to start as the volume is so important that you end up to look for some random patterns that might be suspicious. I wrote a small software called <a href="" >logs-ranking</a> to prefix each line of a log file (currently only W3c (common/combined) logs files are supported) with the ASN and its BGP Ranking value. <a href="" >logs-ranking</a> uses the whois interface of RIPE RIS to get the origin AS for IP address and the CIRCL BGP Ranking whois interface to get the current ranking.</p><p>To use it, you just to stream your log file and specify the log format (apache in this case).</p><pre>cat ../logs/| perl -f apache &gt; </pre><p>and you'll get an output like this with the origin ASN and the ranking (a float value) prefixing the existing log line:</p><pre>AS15169,1.00273578519859,74.125.... AS46664,1.00599888392857,173.242... </pre><p>So now, you'll be able to sort your logs by the most suspicious entries at first (at least from the most suspicious Internet service provider):</p><pre>sort -r -g -t"," -k2 </pre><p>So this can be used to discriminate infected clients from Proxy logs that tries to reach bulletproof hoster where the malware C&amp;C is hosted. Or infected machine on Internet trying to infect your latest web-based software… the ranking can be used for other purposes, it's just a matter of imagination.</p><p>Tags: <a href="" >networkforensic</a> <a href="" >infosec</a> <a href="" >freesoftware</a></p> Sun, 6 Mar 2011 22:09:03 GMT 2011-03-06 Why The Philosophical Works Should Be Free <p> <a href="" title="A close look at a Welsh onion flower by Alexandre Dulaunoy, on Flickr" ><img src="" width="240" height="160" alt="A close look at a Welsh onion flower"></a> </p><p><a class="url http outside" href="" >Roberto Di Cosmo</a> recently published a work called "Manifeste pour une Création Artistique Libre", the work is not really a manifesto in the traditional sense but more a work about the potential licensing scheme at the Internet age. My blog entry is not about the content of the work itself but more about the non-free license used by the author. On the website <a class="url http outside" href="" >many people (including myself) made comments</a> about how strange is to publish a work about free works while the manifesto itself is not free (licensed under the restrictive CC-BY-NC-ND). The author <a class="url http outside" href="" >replies to the questions</a> explaining his rationals to choose the non-free license with an additional "non printing" clause to the CC-BY-NC-ND.</p><p>I have a profound respect to Roberto's works regarding the promotion and support to the free software community but I clearly disagree with the facts stating philosophical works must not have any derivative and cannot be a free work. I also know that Richard Stallman disallows derivative work on his various works. If you carefully check the history of philosophical works, there are a lot of essays from various philosophers having some revision due to external contributions (e.g. Ivan Illich has multiple works evolving over time due to interaction or discussions with people). It's true that the practice was not very common to publish about the evolution of the works. But that was mainly due to the slowness of the publishing mechanisms and not by the works themselves.</p><p>The main argument used to avoid freeing the works is usually the integrity of the author's work. A lot of works have been modified over time to reflect the current use of the language or make a translation to another language. Does this affect the integrity of the author's work? I don't think so. Especially for any free works (including free software) attribution is required in any case. So by default, the author (and the reader) would see the original attribution and the modification over time (recently improved in the free software community by the extensive use of distributed version control system like git).</p><p>Maybe it's now the time to reconsider that free software is going far beyond the simple act of creating software but also touching any act of thinking or creation.</p><p>Tags: <a class="outside tag" title="Tag" rel="tag" href="" >freedom</a> <a class="outside tag" title="Tag" rel="tag" href="" >freesociety</a> <a class="outside tag" title="Tag" rel="tag" href="" >society</a> <a class="outside tag" title="Tag" rel="tag" href="" >copyright</a> <a class="outside tag" title="Tag" rel="tag" href="" >freesoftware</a></p> Sat, 5 Mar 2011 11:18:02 GMT 2011-03-05 Monitoring Memory of Suspicious Processes <h2 id="Heading1">Monitoring The Memory of Suspicious Processes</h2><p>If you are operating many GNU/Linux boxes, it's not uncommon to have issues with some processes leaking memory. It's often the case for long-running processes handling large amount of data and usually using small chunk of memory segment while not freeing them back to the operating system. If you played with the Python <a class="url http outside" href="" >"gc.garbage"</a> or abused the Perl <a class="url http outside" href="" >Scalar::Util::weaken</a> function but to reach that stage, you need to know which processes ate the memory.</p><p>Usually looking for processes eating the memory, you need to have a look at the running process using ps, sar, top, <a class="url http outside" href="" >htop</a>&#x2026; For a first look without installing any additional software, you can use ps with its sorting functionality:</p><pre class="real">%ps -eawwo size,pid,user,command --sort -size | head -20 </pre><pre class="real"> SIZE PID USER COMMAND 224348 32265 www-data /usr/sbin/apache2 -k start 224340 32264 www-data /usr/sbin/apache2 -k start 162444 944 syslog rsyslogd -c4 106000 2229 datas redis-server /etc/redis/redis.conf 56724 31034 datap perl ../../pdns/ 32660 3378 adulau perl daemon --reload 27040 4400 adulau SCREEN 20296 20052 unbound /usr/sbin/unbound ... </pre><p>It's nice to have a sorted list by size but usually the common questions are:</p><ul><li>Is that normal?</li><li>What's the evolution over time?</li><li>Does the value increased or reduced over time?</li><li>Which memory usage is evolving badly?</li></ul><p>My first guess was to get the values above in a file, add a timestamp in front and make a simple awk script to display the evolution and graph it. But before jumping into it, I checked in <a class="url http outside" href="" >Munin</a> if there is a default plugin to do that per process. But there is no default plugin&#x2026; I found one called <a class="url http outside" href="" >multimemory</a> that basically doing that per process. To configure it, you just need to add it as plugin with the processes you want to monitor.</p><pre class="real">[multimemory] env.os linux env.names apache2 perl unbound rsyslogd </pre><p>If you want to test the plugin, you can use:</p><pre class="real">%munin-run multimemory perl.value 104148992 unbound.value 19943424 rsyslogd.value 162444 apache2.value 550055 </pre><p>You can connect to your Munin web page and you'll see the evolution for each monitored process name. After that's just a matter of digging into "<a class="url http outside" href="" >valgrind</a> --leak-check=full" or use your favorite profiling tool for Perl, Ruby or Python.</p><p> <a href="" title="Monitoring Memory of Suspicious Processes by Alexandre Dulaunoy, on Flickr" ><img src="" width="497" height="304" alt="Monitoring Memory of Suspicious Processes" /></a> </p><p>Tags: <a class="outside tag" title="Tag" rel="tag" href="" >unix</a> <a class="outside tag" title="Tag" rel="tag" href="" >command-line</a> <a class="outside tag" title="Tag" rel="tag" href="" >memory</a> <a class="outside tag" title="Tag" rel="tag" href="" >monitoring</a></p> Sat, 1 Jan 2011 11:18:56 GMT 2011-01-01 Often I m wrong but not always <p> <a href="" title="A shaky night by Alexandre Dulaunoy, on Flickr" ><img src="" width="240" height="160" alt="A shaky night" /></a> </p><h2 id="Heading1">Often I'm Wrong But Not Always...</h2><p><strong>Prediction is very difficult, especially if it's about the future.</strong> <em> <a class="url http outside" href="" >Niels Bohr</a></em></p><p>Usually at the beginning of the year, you see all those predictions about the future technology or social comportment in front of those technologies. In the information security field, you see plenty of security companies telling you that there will be much more attacks or those attacks will be diversified targeting your next mobile phone or your next-generation toaster connected to Facebook. Of course! More malware or security issues will pop up especially if you increase the number of devices in the wild, their number of wild users and especially those wild users waiting to get money fast. So I'll leave up to the security companies waiting to make press release about their marketing predictions.</p><p>As we are at the beginning of a new numerical year, I was cleaning up a bit my notes in an old Emacs folder (from 1994 until 2001). I discovered some interesting notes and some drawings and I want to share a specific one with you.</p><p>In my various notes, I discovered an old recurring interest for Wiki-like technologies at that time. Some notes are making references to some Usenet articles (difficult to find back) and some references to <a class="url http outside" href="" > articles</a> how a wiki is well (un)organized. Some notes were unreadable due to the lack of the context for that period <a class="footnote" name="f1" title="note to myself: better explain context when describing something or an idea to implement" href="#1" >1</a>. There is even a mention to the use of a Wiki-like in the enterprise or building a collaborative Wiki website for technical FAQ. There are some more technical notes about the implementation of the software to have a wiki-like FAQ website including a kind of organization by vote. I let you find the today's website doing that&#x2026;</p><p>Suddenly, in the notes, there is a kind of brainstorm discussion about the subject. The notes include some discussion from myself and from other colleagues. And there is an interesting statement about Wiki-like technology from a colleague : <em>it's not because you like the technology that other people will use it or embrace it</em>. That's an interesting point but the argument was used to avoid doing something or invest some times in Wiki-like approach. Yes, this is right but the question is more on how you are making stuff and how people would use it. My notes on that topic ended up with the brainstorm discussion. A kind of choke to me&#x2026;</p><p>What's the catch? Not doing or building something to test it out. You can talk eternally about an idea if it is good or bad. But the only way to know if this is a good or bad idea is to build the idea. I was already thinking like that but I forgot that it happened to me&#x2026; Taking notes is good especially when you learned that you should pursue and transform your ideas in a reality even with the surrounding criticisms.</p><p>My conclusion to those old random notes would be something like this:</p><p><em>If you see something interesting and you get a strong conviction that could succeed in one way or another, <strong>do</strong> or try something with it</em>. (please note the emphasis on the <strong><a class="url http outside" href="" >do</a></strong>)</p><p>Looks like, I'll keep again this advise for the next years&#x2026;</p><p><a class="outside tag" title="Tag" rel="tag" href="" >contribute</a> <a class="outside tag" title="Tag" rel="tag" href="" >startup</a> <a class="outside tag" title="Tag" rel="tag" href="" >innovation</a> <a class="outside tag" title="Tag" rel="tag" href="" >notes</a> <a class="outside tag" title="Tag" rel="tag" href="" >internet</a></p>