Firewall Leak Tester

May 11 2006 : Two new services from Firewall Leak Tester

browser test page added :

1 - a new page displaying your HTTP information (browser test) :

When you browse, you can have the best software/hardware firewalls, the best Antivirus, you still send information out about you. This new page aboutme.php could be called a "browser test", it simply displays the information you send out via the HTTP protocol. The concept is not new, there is plenty of pages of this kind on the Internet, I just thought it was the Firewall Leak Tester's duty to inform you about any data going out your computer.
To the bottom of this page, there is a link to a document giving you all the keys to understand the problematic, and to take decisions as to what to do.

The country and ISP detection may not always be accurate, I do not contact any third party website or database to retrieve these information, I did it the easy way, it is just a wild guess.

2 - a simple page displaying your IP :

Sometimes, you want a simple and quick method to get your IP. It can be by curiosity, or by real need, and it is not always easy to have. For instance, if you are a profesional auditing a network, depending of the type of the audit, you may not have this information. You will now be able to go to the following page getmyip.php to display your IP address.
Still nothing new, but it was so easy to do, that there is no reason not to do it. It might be usefull for someone. Both pages give you your IP address anyway.

If you have any comment to improve these new services, feel free to contact me.

March 23 2006 : scoreboard update

a new statistic table added :

1 - a new statistic table on the tests page :

The current Firewall Leak Tester scoreboard, gives points to the firewalls depending on many things.

First, all of the tests have not the same weight/value, for instance the AWFT test #6 weights more than the AWFT test #1 or even Tooleaky.
Then, depending on how the firewall is handling the leaktest, that it be a clean pass or a generic block, the amount of point given is different.
As a result, a score based on 27 is given, and it gives the score based on the Firewall Leak Tester criteria.

However, while some people are interested in the previous testing criteria, some other are interested to know which leaktests the firewalls are handling (= not failed) no matter how they do, no matter if it is a pass or a block, and no matter the complexity of the leaktest and the supposed weight of it.

A new statistic table has consequently been added, it just take the current results from the scoreboard, and counts the number of leaktest not failed. Thus, it gives a results on 18 (the number of leaktests), regardless of the fact that a firewall can pass in a better way a particular leaktest than another firewall, what only matters in this table, is the leaktests not failed.

The scoreboard stays the base of the Firewall Leak Tester criteria, and stays the base for rewarding the firewalls.
The additional table is added to let you choose with which angle you wish to look at the results, you cannot really compare both.
For instance, a firewall warning about more network accesses (and not cryptic alerts such as 'global low level mouse hook') than another one will have more points on the scoreboard, but the other globaly can handle one additional leaktest and thus have a better score in the new table.

2 - the reward page

The reward page has been reworked to better show the priority of the Firewall Leak Tester rewards. The page now clearly shows that there is one HIPS reward and two firewall ones, and that the HIPS one is the highest reward from Firewall Leak Tester.

Also, the personal firewalls now including (or even becoming) HIPS softwares, I have broken the rule which was doing a difference between firewalls and HIPS, and which said that an HIPS couldn't be on the scoreboard. There will always be the "die hard fans" of the "firewall" being a simple packet filter, that's true for the dedicated standalone firewalls, not anymore for the "personal firewalls" installed on home computers, which include for years now an application filtering, and now HIPS features.
Consequently, firewalls having HIPS features or a full HIPS part (security suite), or HIPS having a firewall part, will be able to be tested on the scoreboard.

The first example of this evolution added to the scoreboard, is Kaspersky Internet Security 6.0 (KIS6), which includes Kaspersky AntiHacker (KAH) and the Proactive Defense (HIPS).

3 - Misc

Althought I do not always participate to every post done on every forum about Firewall Leak Tester, I read them, I compile people's opinions on my side, and I try to understand what people wants to know or need, and not only what I can or want to show. This current update is a consequence of such method, and I hope some people will find here what they were looking for.

I have added at the bottom left a picture to click, to scan your computer with the KAV online scanner. It was a long time ago I thought to do that, but I didn't know which online scanner to choose, and this point was left pending.
Finally I have chosen KAV for it's high well known detection rate, but be aware that the KAV online scanner only detects, and does not repair.
The best thing to know before testing any leaktest, is to ensure that your computer integrity is not already breached, and that you are not infected.

Be aware also that some leaktests are detected as viruses/trojans whereas they are not.

March 12 2006 : "LeakTest" from Steve Gibson updated

just a maintenance update :

Someone has sent me an email pointing out that the version hosted on http://www.grc.com was newer than the version I was hosting, and althought the MD5 and SHA hash I was providing was correct for the file I was hosting, it was obviously different than the newest version from grc, hence being confusing.

I have updated the file I host to the newest version, results should not be different, it is still the same major and minor version (1.2).

As a recall, LeakTest tests if your firewall is using a secure hash to check your applications, or simply a weak path.

Side note for the person who contacted me, my reply to his email came back to me, rejected by his MX server with the response "Service unavailable".

March 11 2006 : leaktest results updated

new 2006 updates:

1 - leaktest scoreboard update

15 firewalls has been tested in this testing session on a Windows XP Home edition, with all updates applied. There is now two result tables, a detailed one, with results per leaktest, and a statistical one, showing the % of leaktest passed.
All of the firewalls were configured at their highest settings before testing them.

Unlike before, you cannot access directly the scoreboard page. By cliking the "tests" link on the left, you access the "test_overview" page, which explains some things needed to understand the results, before reading the results themselves.
Please do not link directly the "tests" page itself, anyone not having passed on the "test_overview" page will be redirected to it anyway.

2 - new leaktests : breakout 1 & 2, Jumper

Breakout 1 (original filename "breakout-en.exe", from Volker Birk) is using the Windows 'SendMessage' API to type the url targetted in the browser url bar. Only Zone Alarm Pro is passing this leaktest at the moment of the tests.

Breakout 2 (original filename "breakout-wp.exe", from Volker Birk) is using the Active Desktop and set an HTML page as a wallpaper, which has for effect to load (and so, to send data out) a remote web page. No personal firewall is passing this leaktest.

Jumper (my new leaktest), is using the Windows Registry to bypass the personal firewalls, especially the AppInit_DLLs registry entry. No personal firewall is passing it at the moment of the tests, althought three of them see "something" with generic sandbox features, like Zone Alarm detecting explorer.exe being killed. In a real trojan, it would wait the next system startup and would not kill any process.
The firewall venfors I am in contact with, has been contacted about it the February 23 2006.

3 - evolved testing criteria : Imunity, generic block, Rank

The Personal Firewalls are evolving, so must the testing criteria. Before, it was either a 'clean' pass, or it was a fail (when some sandbox features were used). Due to the fact that many firewalls are adding to their product sandbox/HIPS features (such as process memory modification protection), when a leaktest is blocked in this way, it gives a half-point (0,5 point) and not anymore 0. Still, the clean 'pass' has more weight (1 point), but at least now 'blocking' a leaktest weights more than seeing nothing and being completely bypassed.

Then, a new test has been added : Immunity. This is not a leaktest, it is just a simple manual verification that the firewall either, cannot be killed (why trying complex things when there is an easier way), or that if it can be, network traffic is still monitored or blocked.

Finally, by now, the firewalls are classified by rank. They are Advanced+, Advanced, High, Medium, and Low. Products classification idea taken from the av-comparatives.org website. The purpose is to group the firewalls per "quality". If you have an "Advanced" firewall, it's not needed to uninstall it to install another Advanced one having only a few more % of leaktest passed.

4 - particular firewall notes (Jetico, L'n'S, KAH, Sygate, PrivateFirewall, Comodo)

Jetico firewall, is the winner of this test session. I'm glad to see such dedication from a firewall vendor to improve their product. Jetico is the first to properly pass my leaktest Wallbreaker. It's GUI and it's architecture are not standard however, and it can be hard to get used to it. Taking in count the overall firewall, it does stateful UDP and TCP inspection, and is very light. Good job to them.

Look'n'Stop firewall has a 20/27 scores. Their lastest beta driver is crashing on x64 CPU with Windows DEP enabled, so I wasn't able to test it. However the driver is able to pass more leaktests : Copycat, PCAudit2, DNSTester. This driver still need some fixing, and was not included in the tests.

Kaspersky Anti-Hacker firewall was removed from the scoreboard, since the next Kaspersky Internet Security 2006's 'Proactive Defense' feature is rewarded by the firewall leak tester "Best Choice" award, and that KAH is included in KIS2006. A product cannot be at the same time on the firewalls scoreboard and on the sandbox reward page.

Sygate firewall has been bought by Symantec and is discontinued. It has consequently been removed from the scoreboard.

PrivateFirewall was a little bugged on my testing computer, and it's 'process control' feature was not activated. I have worked with the firewall vendor to find the solution. PrivateFirewall makers are still working to improve their product, and an update should be done quickly.

Comodo firewall vendor have contacted me, to inform that they are working on the Comodo version 2.0, which has been redone to pass a lot more leaktests. As soon as the new version is out, Comodo results will be updated.

A side note about Norton Personal Firewall, besides his good leaktest score, I found it a big ressource hogger. From all of the firewalls that has been tested on this update, this is unquestionably the one which slow down the much the computer. I am not saying that the product is bad, but simply that you need a powerfull computer to not be impacted too much.

5 - updated pages : categories, softwares, rewards

The Categories page has been updated to take in count the new leaktests Jumper & Breakout, and the category "registry injection" and "Windows messaging" have thus been added.

The Software page has been updated, for few softwares I advise different ones than before, but besides that the look has been partially reworked to make the information easier to access.

HIPS award :

The Reward page now rewards Kaspersky 'Proactive Defense' and Ghost Security Suite. Kaspersky 'Proactive Defense' includes very unique features and is included in both Kaspersky Internet Security 2006, and in Kaspersky AntiVirus 2006. It can detect many things, such as processes hidden by rootkits, or a browser launched with parameters. Ghost Security Suite includes AppDefend (system protection) and RegDefend (registry protection), and can protect your processes against many things, such as Termination.
Please note that Kaspersky 2006 sofwtares is still in final beta stage, and there is no final released version yet.

6 - Misc : backup

Although the leaktests are proof of concept and not real trojans, if they are blocked in the middle of what they were doing, and if not expected in the code, the system can have various unexpected behaviour. Depending on the firewall and the hooks it install on the system, rarely a leaktest can crash, leading from a simple explorer.exe crash to a system halt. For instance I had few issues with PCAudit. Also, if you have no backup and cannot do one, I do not advise you to test the leaktest 'Jumper'. Indeed, this one is playing with the critical AppInit_DLLs registry entry, and even if no problem was found, do not take any risk without a backup.
As with any security "stressing", that it be network flood/stress, Antivirus testing, etc... having a good system backup is strongly advised in any case. Same goes with beta testing softwares or leaktests.

February 06 2006 : news about the site

some news for the year 2006 :

It has been a while now that the leaktests page hasn't been updated. The first reason at first was because on one hand I have been busy, and in the other hand I wanted to let some time elapse before doing the tests again. However a lot more time than foreseen has elapsed (for many reasons) and I would like to not only update the results, but to test more firewalls, and also to update slightly the testing protocol to take in count pure application features.

Unfortunately, my main laptop dedicated to testing the firewalls and leaktests has died, and I'm looking at a new one. If anyone has, or know someone who has, an old laptop supporting Windows XP, and is willing to make a hardware donation, please contact me. The other possibility is to make a donation via PayPal, on the Contact page. If enought people do it, I should be able to buy an old laptop.

About the reward page, I should update it on March or April, with very powerfull and interesting softwares, stay tuned.

Finally, I have received a lot of emails about people warning me of their antivirus detecting viruses on my website. This is not the case, I redirect the readers to the following page "leaktests, viruses or not ?".

Thanks you in advance to anyone willing to help firewall leak tester.

October 16 2005 : minor website update

NEWS : some pages redone, news from firewall vendors :

Firewall Leak Tester is still alive, I had just lately not enought time to update the scoreboard with the lastest firewall versions.

However, even if the results (the scores) are important to have a quick global view of leaktests impact on the firewalls, what matters really is to bring awarness among users and above all among personal firewall vendors. This lastest point, apparently, is globally reached.
Indeed, now some personal firewalls include the leaktests in their developpement stage, such as the beta version of Kaspersky Anti-Hacker, the current Outpost and Look'n'Stop version, as well as Jetico firewall, etc... This point in itself is already a victory, and no matter the user's opinion about the leaktests (he has perfectly the right to disagree with the firewall scoreboard), the final product he will have on his computer will be more secured and more reliable.

I have contacted some of the personal firewall vendors, and below is their point of view about their product, the leaktests, and what work has been done :
Personal Firewalls Vendors point of view
For now there is Outpost and Kaspersky, Look'n'Stop should follow, and probably ZoneAlarm. Any other firewall vendors willing to participate can contact me

About the website itself, it has been globally reworked to make it easier to read, such as the Advices page and some others.
On the left menu, the page Malwares is available by cliking on "In The Wild". This pages shows real malware names using leaktests methods. However, not being into the malware industry, I do not have any databse to rely on to write down here some malwares names. Hence, this page do not shows all of the malwares ITW using leaktest methods, just a few I have found by a quick Google search. If some people have other information, you can contact me at this email address.

I have removed the old "Statistics" page, not having the time to update it. Finally, I have added the freeware ProcX that I have done initially for the website GhostSecurity.

March 1 2005 : ProcessGuard innocent

ProcessGuard & Punkbuster agreement :

Wayne from DiamondCS, ProcessGuard company, has talked with Tony Ray, the head of Punkbuster. Wayne's comments are :

Tony Ray (the head of Punkbuster) was kind enough to put his weekend on hold to reply to my email on Sunday to help clarify the situation, and seems very keen to solve the problem in a friendly and productive manner.

Basically to sum up what he said, yes they have put a specific block on ProcessGuard and they do admit it's an extreme measure to take, but said that this is only temporary until they can come up with a workaround or better solution. I will be assisting Tony in regards to this.

I don't play computer games and I haven't had a chance yet to analyse exactly what they're doing, but as acknowledged by Tony the problem is that Punkbuster seemingly relies on something that Processguard allows you to easily stop - being able to modify other processes (ie. the game that it's protecting) as this is a core part of its protection techniques, which it does at the user-mode level. However, because ProcessGuard does this at the kernel level, user-mode process modifications such as that used by Punkbuster don't really stand a chance, which is why they've taken this measure.

It's certainly an interesting one, and actually isn't the first time this has happened with ProcessGuard. Late last year the "Gunbound" program also temporarily blocked users if it detected Processguard, but that didn't last long after their author received a lot of complaints. Again the problem was that the protection technique they were relying on (inter-process modification) was something that ProcessGuard could easily stop.

The Punkbuster software is essentially about protecting other software (mostly games i believe), but if a feature of a security program (so, not even a specific attack) can defeat this protection technique, then clearly the technique or approach is fundamentally flawed. As a software developer myself I deal with the issue of software integrity (such as protecting against cracking) on a regular basis so I appreciate and respect where they're coming from and the difficulties they face in trying to stop what is virtually impossible (client-side software modification), but like I told Tony I don't believe that customers should be told to turn off security software simply to play a game - this is when protection has gone too far, IMHO. Likewise, it is bad programming practice (and also against most security policies) to inject code into processes without a good reason.

Anyway I hope that sheds a bit more light on the situation, and I look forward to working with Tony to help improve his protection techniques so that not only do they not require users to turn off security software, but also so that they're stronger and more crack-resistant as well, so everyone's a winner.

Tony Ray also replied to me, in regard to the message I have sent to their support :

Hello,

I had your email passed to me. I understood that it had been answered.

We are working with the makers of PG to work out better detection. I appreciate your suggestions and thoroughness. We hope to allow PG to run in future versions of PB without kicking as long as PB itself is not blocked. Hopefully, the issue will be addressed quickly.

thanks,
Tony

So ProcessGuard and Punkbuster will work together to fix this issue, and to finally allow ProcessGuard users to use their security software they have bought, whilst still detecting the cheats which was running under the cover of ProcessGuard. In this way everyone will be the winner.

I'm glad that this problem find a reasonable solution.
Thanks to Tony Ray to have been open to our suggestions and comments.
Hopefully the problem will be solved soon.

February 27 2005 : ProcessGuard blacklisted by video games

ProcessGuard too much powerfull ? :

America's Army is a free popular video game, including Punkbuster, an anti-cheat software done by evenbalance :
http://www.evenbalance.com

After the last Punkbuster update the February 24, everyone having ProcessGuard Installed is automatically kicked from the server without being able to play. You can give FULL allowances to the game executable or even completly disable ProcessGuard protection, as soon as you have it installed, you are kicked. The only solution which works is to uninstall ProcessGuard.

Evenbalance answered to someone else this (I have personally contacted them and I am awaiting their answer) :

At this time you need to choose whether you want to run processguard and/or similar programs or play on PB servers. Both are optional software so it is your choice. I understand if you don't want to close processguard. In that case, you just need to play on non-PB servers.

ProcessGuard has been used by cheaters to prevent Punkbuster to detect them, and Punkbuster instead of checking that it has full access to the OS, simply blacklists ProcessGuard.

I have contacted them to offer a technical solution to still be able to catch the cheats they are after without blacklisting ProcessGuard, I will keep you informed as soon as I will have an answer.

Besides the fact that to play on servers where Punkbuster is disabled is not a choice at all (some people are part of gaiming clans where tournaments are required to take place on Punkbuster-enabled servers), the whole problem here, is that while on one hand we are trying to convince people to protect them with security softwares, on the other hand some companies tell them to disable, or in our case, to uninstall their security softwares.

That's not a tolerable situation (no matter they have the right to do so or not).

December 31 2004 : new page added to the website

page about in the wild malwares and leaktests exploit use :

I am sure that by reading this site the reader feel concerned about whether or not these various demonstrated exploits can possibily a day hurt him.
To try to answer to this question, I have added a small article talking about this,
available HERE. That is probably not as much a definitive answer as 'yes' or 'no', but I hope it will present the facts in a more real and concrete manner.

I take this occasion to whish you all happy holidays, and a very good happy new year with all my best whishes. Do not forget that some people in Asia need our help, do not hesitate to donate to aids organizations to help them, for that they can also live decently the begining of the new year.

November 1 2004 : ProcessGuard v3.0 released

new ProcessGuard version :

ProcessGuard v3.0 is now out, and includes many changes.

Usually I do not add in the news page new releases of products that it be firewall or any other security software. However, I believe that ProcessGuard is a unique and powerfull application which can block any kind of leaktests and malware.

The new version protects against the \Device\PhysicalMemory exploit which can disable any security software. In addition the Close Message Handling has been improved and enables the user to define part from a program for which a popup (human HID) should appear (e.g File menu -> Exit) to avoid malwares to send keystrokes to close your security software.

To sume up, ProcessGuard can protect you from process termination, code modification, Rootkits, leaktests, password stealers, process-injecting trojans, keystroke loggers, windows file protection disabling, user imitation, physical memory exploit, hooks, etc...

In addition to any personal firewall, ProcessGuard increases dramatically your security by covering what your firewall cannot handle properly.

A quick review is available here, or you can download it at the official website.

October 13 2004 : Major website update

NEWS : many updates and improvments :

1 - First, the Test page has been updated and now test two additional firewalls, Jetico and Kaspersky Anti-Hacker. The tests has been more difficult and time consuming than ever, due to the firewall evolution toward what we would call firewall/sandbox hybrids.
That's why the results shown for your favorite firewall may not be what you were expecting, due to few sandbox features I am disabling to let the outbound application filtering component alone against the leaktests.
More information about how I am doing the tests in the Document page.

Some notes and points : as an exception to the rule, Jetico firewall which is still beta (and only beta, no final or stable versions exists) has been added, because it seems promising. However I did notice bugs that I have sent to their dev team, and I do not advise to use it now on a production computer. Then the results of this firewall will be really different from what you may find yourself, which is due to their full "parent based" technology, which is indeed efficient but can only be seen as a makeshift in my tests. However as I said it seems promising, let's see what it turn to in the future.

Then, I have added the Zone Alarm 5.5 beta version, because it's their last available version, and I wanted to test their lastest product.

I also want to point out that Outpost has done a big jump in the score table, and was very close to simply have the best results.
Agnitum is one of the firm who has the most demonstrated his dedication to leaktest protection, and listened many of my explanation, for instance for Tooleaky and the general launcher leaktest category which is now correctly handled.
A special "dedication reward" for Agnitum so.

From Agnitum :
Firewall Leaktester project provides users with very valuable information about current outbound security breaches that allow private data to be sent from users computers. Agnitum is aware of all recent security breaches putting our best efforts update Outpost Firewall in order to protect against all online threats.
According to many independent security experts Outpost Firewall Pro is one of the few personal firewalls that can secure against virtually all possible outbound and inbound methods of bypassing firewall protection. This fact makes Outpost "the go to" application to protect any personal computer with Windows operating system.

They have seriously studied the leaktests, and have made available their results, explaining what Outpost was passing from their point of view and why.
This document is available There

About Norton firewall 2005, I found it really heavy on my poor laptop, in fact it is the heaviest firewall I have tested ressources speaking. It should run fine on a high end computer, but I do not advise to run it on a 500Mhz computer.

Finally, mostly for time reasons, I do not include anymore the Win9x/Me and "out of the box" results, more explanations on the test page itself.

2 - A new reward system has been added, which reward both the personal firewalls and the sandboxes softwares.
You can see on the left menu the "4 - Rewards" link which links to the rewards themselves.
There is a Gold and a Silver reward, why not a bronze reward ? Because it is not like the Olympic games where you can have only one person on one step of the podium, so 3 medals in total. Here instead, many firewalls can be Gold, and many can be Silver (if they have the same score) which already could reward 4 firewalls or even 5 if the results are really close, 5 rewards with 2 medals, imagine with 3 medals, almost all the firewalls would be rewarded which would be sensless and meaningless.

For the sandboxes there is only one reward, the "Best choice" medal. More details on the reward page.
For now about Process Guard, it is the v3 beta which is rewarded but I will update as soon as the final is released.

The logos/rewards has been designed and made from scratch by Jade (aka Bowserman), many thanks to him !

3 - From now I am stating clearly my adoption of the "responsible disclosure" althought it does not really apply to me because the leaktests do not show firewall's vulnerabilities in the usual security sense, but show firewalls weaknesses which is not the same.
A firewall's vulnerability will allow an attacker to crash it, to execute arbitrary code via a buffer overflow, to escalate system priviledges, to do a Denial Of Service (DoS), whereas a weakness will only allow a malware to escape the detection without attacking the firewall (at least that is the difference for me, althought for some people it might still be a vulnerability).
Anyway I am using this disclosure policy which I think is very fair. To sume up, 5 days to contact the firewall vendor, then we agree of a time frame, and after the deadline the leaktest is released, that an update is available or not.

Of course I have no power and no influence on the release of a leaktest that I did not do and which is released by someone else.

Finally I must say that I was very surprised by the excellent firewall scores, which demonstrates that the firewall vendors has been listening what I was saying about the leaktests, many contacted me to have more details about my criteria, and how to secure more their product.
At the end the end user is the winner, having a better firewall, that he agrees or not with the leaktests testing protocol.

For sure there will always be whatever the subject people agreeing or disagreeing, and that's why the leaktests themselves are downloadble for you can test yourself your firewall following your personal criteria.

To finish this news, the site costs me money (webhoster + laptop bought to do the tests) and if you whish to support the website to pay this as well as firewall licences, or simply just to support me, you can donate if you whish so, see the contact page for this.

October 9 2004 : Wallbreaker v4.0 released

new Wallbreaker version with a fourth test :

Wallbreaker v4.0 is now out, and includes two main changes.

First, it adds a fourth test which, still staying in the idea of an executable launching another one which launches another one..., uses now the Windows scheduler (at.exe) which in turn execute a task using svchost. Nearly any information could be transmited like this.

Secondly, and it's the most important for me, Wallbreaker targeted webpage is now my "firewallleaktester.com" domain and not anymore the old free webhosting space one, which belongs to my ISP and will die when I will leave them. So now, I have finished my switch, nothing points to the old location.

As I have written on the target page Wallbreaker reaches if it goes out of your computer, if you are vulnerable to it you should read the Advices page and the Software page, and above all install a sandbox such as Process Guard to control anything launching on your system.

September 26 2004 : RSS support

extension for Mozilla browsers and more :

From www.mozilla.org : RSS (an acronym for "Rich Site Summary" and later coined "Really Simple Syndication") is a way for web sites to summarize their content, such as news articles, to make it available in a different view.

RSS allows you to read the headlines of a website from a RSS software reader, and now from Mozilla browsers as well, such as the Mozilla FireFox browser.

This feature is highly interesting for news sites such as Slahdot, but can still be handy for any site having news, that you can merge all together in the same bookmark folder, to see quickly what have changed and what is new without browsing all of the websites.

The Mozilla FireFox browser users will see at the bottom right this icon

, and can click on it to add the website news in their bookmarks. With Mozilla FireFox you can right click on the bookmark folder and select "refresh live bookmark" or use the "sage" extension to update your RSSs.

For now only the last news are on it as a test, but if you have any suggestion about what else would be valuable to add, you can send your ideas to me using the online form.

For direct news in your inbox, you can prefer the mailing list.

September 12 2004 : Atelier Web Tester v3.1 update

new version released :

Atelier Web Firewall Tester (AWFT) 3.01 has been updated to the 3.1 version, and is available here.

From the 'release.txt', the modifications are :
All tests have been reviewed for better performance

Basically the results are still the same, but the GUI is reworked and the tests timeout (when the firewall is blocking) reduced.

In addition I have quoted on his page the description of the 6 tests it is doing, coming directly from the author's website.

September 8 2004 : Steve Gibson *interview*

Interview of Steve Gibson, Gibson Research Corporation :

Mr Steve Gibson, security expert, has kindly accepted to participate to an interview for the site firewallleaktester. The interview is about the leaktests of course, but more generally about Windows and security, how the leaktests can be defeated and what is the point of view of Steve Gibson about all of that.

The full interview is available HERE.

Thanks to him for his time.

September 6 2004 : Anti-Leaktest Guide document

new document published :

The Anti-Leaktest Guide document is commenting more in detail what I mean by 'to pass' and by 'to block' a leaktest, and so why I do a difference between a firewall pass and a sandbox block.

This document tries to explain in a first part for each leaktest what is a pass for it (to be 'passed' in the score board), and in a second part how to block it.

Please note, as said at the end of the document itself, that unlike my previous paper, I will update this document from time to time regarding the users questions, the details needed, and the new firewall results.

About the news page, now you can access the news directly by their number. For instance, the link of this news is http://www.firewallleaktester.com/news.htm#39

August 20 2004 : Windows XP SP2 Firewall tested

What does it worth against the leaktests ? :

The long awaited Service Pack 2 for Windows XP sees along it a rebuilt firewall, better than the SP1 one.

However, althought bringing to the user popups to the screen about applications, XP SP2 firewall still stay an inbound firewall only. This can be confusing because usually when a personal firewall warns the user about an application, it's about an outbound traffic. At the opposite, SP2 firewall warns the user when an application binded to the network interface (listening on a port) is receiving traffic. In this way, using IE will not bring any popup, whereas using a software acting like a server (P2P softwares, IM file transfer, etc...) will.

For the users having no firewall at all, it's better than nothing and provides a good inbound protection against unsolicited connection attempts.

However, for the other users, unlike what I have read sometimes, dumping your current personal firewall to replace it by the SP2 firewall is not a good idea, SP2 firewall does not have outbound protection, and does not protect against a single leaktest.

It's a good step from Microsoft, but not an option if you have already a personal firewall.

The test page has been updated to add SP2 firewall only, nothing else is changed.

August 2 2004 : New leaktest : Surfer v1.1

Surfer v1.1 available Here.

This new leaktest from Jarkko Turkulainen (DNStester author) while still being a launcher (existing categories) is using a new approach by using the DDE inter-process protocol.

Unfortunaly I don't have currently the time to test it against the firewalls.
Next time I will update the test page, it will be with XP firewall SP2, Outpost 2.5 (when it will be released), ZA 5, and all of the lastest firewall builds.

July 29 2004 : Ghost v1.1

Ghost v1.1 available Here.

This new version is NOT an enhanced leaktest, the previous results against the firewalls are still the same.

What has been modified is the target url from my previous free webhoster to my new one firstly, and secondly, the user has now to enter a string of his choice which will be sent to a dynamic webpage and displayed to show the proof that information has been transmitted.

- do not enter any private or personal information, rather "neutral" words such as house, dog, etc...
- data recorded on the database will be automatically and periodially deleted
- the database can be flushed on demand if requested to me for a valid reason
- the page is viewable HERE, please note that this link submit the word "view" in order to display the page, you can't access it without submitting a string.
- the maximum length allowed for the string submitted is of 20 characters only.

This way it is more understandable that information has been transmitted, instead of just displaying a webpage.

If you have any question, you can contact me by email or by using the online form

July 03 2004 : new page : FAQ

The site raising often a lot of questions, sometimes not clearly answered, I have decided to add a FAQ which will be often updated regarding all of the relevant questions that can be asked.

I won't add a news for every FAQ updates, just this time for the creation of the page, so keep visiting from time to time or better, if you have a question not being in the FAQ, submit it to me.

Page available HERE.

June 29 2004 : correction in the DNStester page

The DNStester executable pointed by the link on the DNStester page was good, but not the one pointing on the source code, because this one was the DNSshell source code and not the DNStester one.

The previous dnstest.zip is DNSshell (it's a project from the same author) and dnstester.zip is this time the DNStester source code.

I have so updated the "sourcecode.doc" (zip/7z) document on the document page with the right source code.

June 14 2004 : website re-organized

I have organized the site differently, in a way that making now what you are looking for easier to find.

All contact/online form and mailing list are on the top, which let more space on the left menu. Consequently I have added Windows Worms Doors Cleaner page on it which will avoid to the user to dig into the site to find it.

I am opened if you have any suggestion.

June 12 2004 : mailing list created

A mailing list has been created and is available at the following Page

For now the mailing list suscription page is only available from the Contact page, but the website will be a little modified soon to be better organized.

June 1 2004 : contact page + wwdc manual

The contact page has been updated with the good email addresses to use.
In addition, an Online Form is now available and enables you to send me an email from the website itself without using a mail client.

The Windows Worms Doors Cleaner page now has an additional link to a small help page describing all his features that can be used by the home users, as well as by the administrators.

May 28 2004 : new webhoster + many updates

The website has now definitly moved on http://www.firewallleaktester.com hosted by a new webhoster more reliable and faster.
This webhoster will allow me to add many features to the website like, may be, a mailing list. I don't know for now what will be done, but the site is not anymore restricted.

Consequently I had to update Windows Worms Doors Cleaner 1.4 to 1.4.1 http://www.firewallleaktester.com/wwdc.htm at least for updating the links on the GUI.

Finally, I publish my leaktest paper which has taken more time than foreseen to publish. The text in itself was ready two month ago, but I had to add the last leaktest, to give it for correction to a native english to correct mistakes, I had too to ask to different people what to modified, etc...
Moreover, in the meantime, new security softwares has been released or improved (like Process Guard) and I have added a note about them in the paper, in the advises part.
I hope it will be for help for some people, the description and download are available on the Documents page.

The Contact page has been updated too, with the new email address to send emails to.

The previous free statistics javascript has been removed, sometimes it wasn't working completly and was slowing down the loading of the page.

Many website display bugs related to Internet Explorer has been fixed.
Now the "br" tags are taken in count in both IE and Mozilla, which wasn't the case before with Internet Explorer.
The bug was related to the "letter-spacing" option in a CSS script, if any webmaster has the same problem, contact me by email for the solution

At the end, don't forget to update all your bookmarks, in your browser, and the future links you will give to other people.
I will let alive the old address for at least one month or two, for letting people the time to switch.
The main pages on the old adresses are directly redirected here, but not the others.

May 19 2004 : new page about leaktests flagged as viruses

Many users has reported that their AntiVirus software is detecting leaktests downloaded from this site as malwares.
Of course there is no risk at all, everything is explained on this page

May 18 2004 : updates

Windows Worms Doors Cleaner 1.4 has been released, and is available Here
Please read the warnings included in the changelog about disabling services.

Now, a PayPal button is available on the Contact page if you whish to make donation to support the website and to help to do more tests (e.g : to buy firewall licences)
with

May 05 2004 : updates

The statistics page has been updated with the total of 202 answers from the poll.

Apart of that, many IE users told me the left menu wasn't showing great, this is because the site is above all made to be the most compatible with Mozilla FireFox ( screenshot )

April 26 2004 : Windows Worms Doors Cleaner 1.3

Added support for UPNP and Messenger service (enable/disable) vulnerabilities, full changelog there :

version 1.3

Page modified and updated to describe each vulnerability.

If you find any problem with it, please email me for i can do the necessary modifications

While i'm here, i say again that the site is by far better viewable with Mozilla browsers such as FireFox (Internet Explorer drops lines return "br" tags on the site)

April 14 2004 : Windows Worms Doors Cleaner 1.2

Windows 2003 server support added and more, full changelog there :

version 1.2

All of the improvments from 1.0 to 1.2 are thanks to users ideas and suggestions, so if you have good ones too, feel free to send me an email, i will do my best to do it if i find the idea valuable.

April 1 2004 : new leaktest : DNSTESTER

new leaktest 'DNStester' released, from the author Jarkko Turkulainen. more information on his DNStester PAGE and on the Categories page

Results page updated, only ZA Pro last version seems to pass sucessfully this leaktests.

Jarkko Turkulainen is the author too of 'DNSshell', an svchost injector, more information on the author's website.

=> In order to use DNStester, you must left enabled the DNS client windows service.

March 25 2004 : updates

Windows Worms Doors Cleaner v1.1 is out, i take care in it of many users suggestions and idea, to make it more understandbale and easy to use, thanks to them.

version 1.1

I have updated too the software page with the new Process Guard 2.000 which now have a dedicated page (link available on the 'software' page)

March 24 2004 : updates

In my efforts, besides leaktests themselves, to help people to secure their computer (as you can see in the 'advices' page or even the 'software' one) i have done a small tool which can be usefull to disable common infection vectors used by worms, the dedicated page is available here :

Windows Worms Doors Cleaner

Another version with small GUI changes is in progress, to see better what is enable and what is not in an easier way.

Is updated too the 'statistics' page, will be updated again when i will have more poll (+200).

March 14 2004 : PCAudit leaktest update

PCAudit v2 goes from internal version v4.0.0.0 to v4.0.1.0

I don't expect results to change, may be the executable was just modified to explain better who should use their test (the text has been modified) or may be the executable was slightly modified to avoid to be caught by Anti-Virus signatures.
The previous version was detected at least by Norton Antivirus as a Spyware (althought it is not).

I think results will remain unchanged, i will do tests again when i will have time. In the meantime, i will ask to Outpost beta tester to test the new version, if Outpost passes it well as well as the prvious version, it is probbale that it will be the same for ZA and NPF.

But as usual, readers are invited to test themselves.

February 27 2004 : Outpost beta tester

Agnitum has suggested me to be beta tester for them, and i have accepted. I will be involved while beta tests only regarding leaktests issues.
I'm happy that a firewall vendor takes so seriously leaktest issue, even if all can't be improved in one day, at least they are doing their best to.

Results won't be biased thought, i am still staying objective, i am only there to help them.
I am open to any other offer from any other firewall vendor too.

February 24 2004 : Agnitum contact

Agnitum (Outpost Pro firewall vendor) has contacted me to offer me a _free_ unlimited licence key file for i can use the full registered Outpost Pro 2.1 version. They have asked my opinion at the same time about their firewall, what i think they could do to improve their firewall, so at the end to protect better their customers. My suggestions has been forwarded to devs.

Agnitum seems so to take care of their customers security and really work hard to fix leaktests issues.

I thought such behaviour should be known, this is why i write it, there is few firewall's vendors who don't care at all of anything else than money.

I thanks them for such generous offers, and i hope to see leaktests improvments in next releases.

February 23 2004 : Results page updated

Results updated with Kerio 4.0.11 and Outpost Pro 2.1
Outpost passes more leaktests than 2.0.x, and an error was corrected regarding Tooleaky and Kerio.

AWFT leaktest goes from 3.0 to 3.01, the modification from authors is :

22nd February 2004 Release 3.01 Fixes: * Unbelievable, but some firewalls are using a fingerprint of AWFT in order to present better results... This release is a temporary workaround (until we find time to a full review of this software).

None of the firewalls i am evaluating seems to "cheat" (same results with 3.01) but i have my idea about which is may be doing this... more news i hope when AWFT author(s) will have fully reviewed the suspicious firewall.

February 20 2004 : ** Major site update **

totally new look thanks to my friend "ibas", the website is now viewable in 800x600
but his best viewed in 1024x768 with mozilla FireFox.

Thanks again to IBAS for his hard work !
(his email adress or/and website available soon)

December 14 2003 : ** Major site update **

- The results page has been updated
- Now results are explained in an additional page (link on the results page)
- Leaktest "Wallbreaker v3.0" released
- New Leaktest "Ghost v1.0" released
- Categories page updated
- Firms pages updated
- all leaktest information pages updated, added : the leaktest categories, leaktests fingerprint (MD5/SHA-1/SHA256)
- new "statistics" page with the most loved firewalls from users.

And soon a total rework of the website graphism to make it more user friendly.

if you find any mistakes in the results page, and that in addition you perfectly understood on which criteria leaktests testing are based on, so that you can be sure of the mistake, pls write an email to gkweb@wanadoo.fr with a subject like
[FWLT, Results error report] : OS/firewallname/leaktestname.
Then, in your mail, please detail your settings and all your firewall configuration (why not with screenshots) which will help me to reproduce exactly the result you found, i don't correct a result only by trust, i must be able to do it myself.

All results has been tested many times each.
However, i apologize in advance if you find any mistake, all that is necessary to be done will be in the purpose to have right and fair results for all firewall evaluated.

November 28 2003 : new page added : security software suite

In the left menu under "advices" page, i added "software" page which show an example of a complete security software set/suite to secure your computer.
This page should be read only after the "advices" one.

November 25 2003 : leaktest added

The leaktest "MBtest" was added.
Carefully read instruction about how to use it.

November 23 2003 : 2 news

1)I added the new leaktest PCAudit version 2, which bypasses firewall in a different way than his previous version.

2)i will do again tests regarding leaktests VS most recent firewall in December 2003 or the later in January 2004.
While this time, you can as usual test yourself your security with provided leaktest if you are sure of understand how to do them, and what results means.

August 30 2003 : WallBreaker v2.1 available

WallBreaker 2.1 has no new features, it is just totally rewritten from C++ to PureBasic, and now the source code is available.

August 10 2003 : Advices page available

I finally added a page about best guidance settings and behaviour, i thanks JV Morris for his huge help.

July 21 2003 : Leaktest results updated

June 18 2003 : WallBreaker v2.0 + results updated

Why an other version one day after the other ? because windows is leaking from everywhere and i discovered an other trick to exploit to bypass firewall, this time, it just use directly internet explorer, and firewall doesn't see nothing...
On chart results, Tiny Personal Firewall has been removed (like Black Ice before) because of the impossibility to test his software filtering fairly because of his sandbox based.
Sandbox are powerfull feature but are not software filtering at network layer However, i still continue to investigate BlackIce and Tiny... to followed.

June 18 2003 : New Leaktest available !! WallBreaker

I finally released my own leaktest : WallBreaker which actually go through all famous personal firewall.
Please be sure to read the "how to do tests" at the bottom of results page.

June 13 2003 : results board updated

New results on the board, but i need now Mac afee personal firewall plus results, and Win 9x/Millenium results.

June 12 2003 : new results board

The results page has been updated, feel free to give your suggestions/comments, i hope you will like it.

June 10 2003 : leaktest removed

A leaktest which was not official and could be used to cause damage regarding the law to his author has beed removed from the website. I will not give more information, it's not my purpose, the purpose it's to follow the law, and due to localized law of author state, i must remove it, for the safe of the author. Hope you understand.

June 7 2003 : new results

new results added for outpost personal firewall. Notice that FireHole is shown as good, but it is detected on WinXP (may be 2000 too) but not on Windows Millenium
If you want to discuss about that, don't hesitate to write yout opinion on the forum.

June 6 2003 : Website

Website still builting...

Home News Contact Online form Mailing list