Recommended mail anti-spam solution: ASSP

March 2, 2009 at 9:43 AMAndre Loker

If you’re running you’re own mail server and looking for an anti-spam solution be sure to take a closer look at ASSP (which stands for AntiSpam SMTP Proxy). I’m using it with my own mail servers and what can I say – I’m more than happy with it.

image

The pros:

  • It’s free, so no risk to try it whatsoever
  • It uses a plethora of different spam recognition and fighting techniques, such as:
    • Greylisting (which alone already keeps off 80% and more of unwanted mass mail)
    • Bayesian filter
    • Sender/HELO validation
    • DNS/address blacklisting
    • Sender Policy Framework (SPF) / Sender Rewriting Scheme (SRS)
  • It’s low on CPU use
  • Choice between test mode (e.g. tag subject of possible spam with something like [SPAM]) or “real” mode (block spam altogether)
  • Lots of options and statistics, all accessible through a browser interface

There are some minor cons, though

  • Installation not trivial
  • Memory use is not spectacularly low (about 50MB on a machine receiving 1000-2000 mails per day)

I don’t find those issues to be real problems. Once it is installed it runs fine and my servers are able to handle the additional memory use.

Here’s what you need:

Conclusion


For me ASSP has cut down the amount of spam tremendously! I’m running in test mode for two month now. Currently I receive between 0 and 7 mails per day tagged as [SPAM], the rest is blocked by greylisting already.

Some useful links regarding ASSP:

Posted in: Tools

Tags: ,

First attempts with Gallio

February 14, 2009 at 2:21 PMAndre Loker

From time to time I like to check my development workflow and tools to see whether I’m still up to date. This time I thought it might be a nice idea to give Gallio a shot. If you don’t now Gallio: it is a platform to uniformly run unit tests from whatever unit test framework (MbUnit, NUnit, MSTest etc.) you use. At least that’s how I understand Gallio’s mission. It also integrates into a bunch of other tools which I use (CruiseControl.NET, NAnt, NCover, ReSharper et al).

So basically it sounds like a cool idea. On top of it, version 3 of MbUnit – my favourite unit testing framework – is available only bundled with Gallio, that's why decided to give it try. Until I only used MbUnit 2.4.

And here’s a list of twitter-esque thoughts about Gallio that came up during my first 60 minutes of use.

  • Installation is no problem
  • Migrating from MbUnit 2.4 to MbUnit 3 didn’t take much effort, at least for the small project I chose for my Gallio evaluation. Mainly I had to replace the RowTest attribute with a conventional Test attribute, because row tests have been consolidated into the normal tests. But you can also use the MbUnit.Compatibility Assembly and don’t change the existing tests at all. Big plus!
  • The documentation is rather incomplete. There are still a lot of TODOs. Especially a migration guide for MbUnit 2 users would be really useful.
  • Integration into Testdriven.NET works well, no problems here…
  • … although the test runner feels a little less snappy than MbUnit 2.4, startup time is slightly higher. The 190 unit tests in the small test project took about 5,2 seconds using MbUnit 2.4 and 7 seconds using MbUnit. I’ve yet to test how this scales with more tests.
  • ReSharper integration seems to be incomplete and instable. If I use MbUnit 2.4 tests, not all tests are run. Only about 30 of my tests appeared in the unit test browser. Using MbUnit 3, all tests were found, but occasionally ReSharper through exceptions when I selected failed tests.
  • Using Testdriven.NET’s “Test with”=>”Gallio Icarus” context menu action fires up Icarus (the Gallio test running GUI) but doesn’t load the assembly of the project that I want to test.
  • Running tests in Icarus is unbearably slow. The tests took more than 40 seconds to run (compare that to the 7 seconds using Testdriven.NET or even 5 seconds using MbUnit 2.4 + Testdriven.NET).
    image
    I guess this is due to the fact that the result window is refreshed after every single unit tests. Thus, the performance even degrades with every additional unit tests. I wonder what happened if I were to run tests of my bigger projects.
  • The one real showstopper however is the fact that suddenly unit tests begin to fail that have been perfectly valid before. This happens when I use Rhino.Mocks to create a partial mock and expect for an ArgumentNullException to be thrown in the ctor of the partially mocked type. Luckily this is easily reproducible:

Class under test:

   1: public abstract class SomeClass {
   2:   // Throws ArgumentNullException if value is null
   3:   protected SomeClass(string value) {
   4:     if (value == null) {
   5:       throw new ArgumentNullException("value");
   6:     }
   7:   }
   8: }

The unit test:

   1: [Test, ExpectedArgumentNullException]
   2: public void CtorShouldThrowIfArgumentIsNull() {
   3:   var mocks = new MockRepository();
   4:   mocks.PartialMock<SomeClass>(default(string));
   5: }

Results when run under MbUnit 2.4

image

Results when run under MbUnit 3

image

A System.ArgumentNullException is expected, but instead a System.ArgumentNullException has been thrown. Possibly it’s a framework version conflict, I don’t know. Anyway, a bug report is on its way (uodate 02/20/2009: the bug report has been closed as won’t fix).

My personal conclusion for now

I really appreciate the idea behind Gallio to unify unit testing. Different projects use different unit testing frameworks. Especially in connection with automated builds it could be useful to have one test runner that supports them all. And of course, I really like that they have chosen MbUnit as their primary unit testing framework, because I think MbUnit rocks and is superior to the rather basic NUnit. I haven’t looked much into the version 3 API of MbUnit, but I think there are some nifty new features that can be really useful in daily life - I’d really like to use them.

But no matter I sympathise with this project, it has some major flaws: lack of documentation, a sluggish GUI, perfectly valid tests that fail, integration issues. All those things sum converge into a situation that makes Gallio not really usable for me as it is now.

Nevertheless Gallio is very promising. I will definitively come back to it from time to time to see how it improves. To the Gallio developers: you’re doing the right thing, keep it up!

BTW: The Gallio people have established a merchandising shop, which I think is intended to support the project moneywise. I think it’s a cool way to support a project and still get something back. But please don’t forget to put the major part of you energy into the project itself, not the merchandising!

Posted in: Tools

Tags: , ,

Simple AOP: introduction to AOP

February 13, 2009 at 12:46 PMAndre Loker

Aspect oriented programming (AOP) allows us to keep implement different concerns in isolation. In this article series I’ll describe ways to make use of AOP without much hassle. This first article serves as an introduction to AOP to those that are not familiar with aspect orientation.

Why AOP

If we analyse the source code software system we will soon realize that the different parts of the code can be grouped according to their responsibilities. For example some parts of the code are responsible for implementing business logic, other parts manage transactions, yet other parts are used to handle exceptions etc. In AOP terms those responsibilities are called concerns. The business logic is considered the core concern of the application, whereas the latter examples represent aspects that somehow interact with the core concerns. Because they interact with the core concerns, aspects are often called cross-cutting concerns. When we look closer at those interactions we can identify two problems:

1. Scattering

Scattering occurs if the same aspect has impact at many different places in the code. For example, logging or transaction management are often used at many different places in the code, but each time almost exactly the same way. This is problematic because it is harder to maintain scattered functionality.

2. Tangle

The second problem that often occurs is that of tangling concerns. At a given place in code – let’s say a single method – different concerns are present and interleaved. To give you an example, here’s a method that has the core responsibility to change the email address of a user identified by some  ID. Three concerns can be identified: the core concern (changing the email address) and two additional aspects (exception handling and transaction management). I highlighted the different concerns using different colours.

tangle

Tangling concerns are a problem because they bloat the code and distract from the core concerns. Here’s the same core concern without any extra aspects:

   1: public Result ChangeEmailAddressBusinessOnly(int userID, string newEmailAddress) {
   2:   var user = Users.GetUserByID(userID);
   3:   if (user != null) {
   4:     user.ChangeEmailAddress(newEmailAddress);
   5:     return Result.Success;
   6:   }
   7:   return Result.Error("User not found");
   8: }

Tangling also makes unit testing more difficult, because all the extra non-core concerns have to be considered during the tests as well, at least they have to be stubbed or mocked.

Using AOP to solve scattering and tangle

AOP tries to solve the two problems using the separation of concerns paradigm. With AOP the different aspects are described and implemented in isolation. Because the behaviour is defined once only and than applied automatically at all places of interest in the code, scattering won’t appear. And secondly, because aspects are defined in isolation the core concerns can be implemented without any intermingling aspects which solves the problem of code tangle.

Advices

The implementation of the behaviour is called the advice of the concern. Here’s an example of the advice belonging to the exception handling concern in pseudocode:

   1: someAdvice(context) {
   2:   try {
   3:     context.Proceed();
   4:   } catch(Exception e) {
   5:     if(ExceptionHandler.ShouldRethrow(e)) {
   6:       throw;
   7:     }
   8:   }
   9: }

In this example context.Proceed() would execute whatever has to be executed at the point where this concern is present. If that code throws an exception we can handle it.

Pointcuts

We must of course somehow define where the concern is present. This is defined by the so-called pointcut. A pointcut is a boolean expression answering the question “should this concern be applied here?” for every possible “here”. Now, what’s a “here”? A “here" in this context is every possible point in the code where a concern can possibly be applied. Those points are called join points. Examples are: “A call to a method named X” or “The assignment of a value to a variable”. What kinds of join points are supported depends on the AOP implementation and is defined in that implementation’s join point model. Adding pointcuts to our pseudocode example could look like this:

   1: pointcut ServiceMethods : call to any method of class Service
   2:  
   3: someAdvice(context) at ServiceMethods {
   4:   try {
   5:     context.Proceed();
   6:   } catch(Exception e) {
   7:     if(ExceptionHandler.ShouldRethrow(e)) {
   8:       throw;
   9:     }
  10:   }
  11: }

Now everytime we call a method of class  Service the advice will be executed. The call to Proceed() will in this case execute the method that is being called.

Advice types

Now that we have defined what is advised and where we can add the minor detail of when it is applied, because we have different kinds of advices. The example above uses a so called around advice. That is, the advice is wrapped around the joinpoint. It controls the moment when the joinpoint is executed explicitly by calling Proceed(). Other examples of advice types are:

  • before and after advices, which are executed before or after the execution of the joinpoint respectively. Those advices can’t explicitly define when the joinpoint is executed.
  • throwing advices, which are only executed if the execution of the joinpoint raised an exception
  • returning advices, which are only executed if the execution of the joinpoint did not raise an exception

Finally, here’s a fine-tuned version of our pseudo code example:

   1: pointcut ServiceMethods : call( Service.* )
   2:  
   3: around(context) : ServiceMethods {
   4:   try {
   5:     context.Proceed();
   6:   } catch(Exception e) {
   7:     if(ExceptionHandler.ShouldRethrow(e)) {
   8:       throw;
   9:     }
  10:   }
  11: }

By the way, I didn’t come up with this notation all by myself. It resembles the syntax used with AspectJ, which is more or less the most prominent (Java) AOP framework available. It has grown quite mature and if you’re interested in AOP I highly recommend to have a look at it. Yes - even it uses Java :-) 

Weaving

So, we have defined a bunch of aspects with their pointcuts and advices. The missing link is obvious: how do those advices get applied to the base code (the code containing core concerns)?  For this process – called weaving - several approaches exist, all with different pros and cons.

Compile time weaving

This approach is taken by AspectJ for example. An additional step is taken during compilation that inserts the advices into the source code or an existing binary assembly. This results in binaries that don’t look any different from conventional binaries.

Pros

  • the join point model can be very detailed (e.g. the assignment of a value to a variable)
  • performance wise there’s little overhead

Cons

  • weaving is static only, we can’t change aspects at runtime

Runtime weaving

With runtime weaving the aspects are woven into the base code – well – at runtime. There are different approaches to achieve this. Some implementations for example modify the runtime environment to support injection of code. A simpler approach is the creation of runtime proxies for advised objects.

Pros

  • no need for an extra compiler
  • aspects can be added or removed dynamically

Cons

  • depending on the implementation, the join point model can be rather limited

For this article series we’ll use proxy based runtime weaving: it is simple and in many cases sufficient.

Critical view

In principle AOP sounds promising: separation of concerns helps to concentrate on core concerns and keeps our code free of tangle and scattering. Where there’s light, there’s also shadow they say and this counts for AOP. If you look at the base code you can’t see at a glimpse which aspects are applied (except if you have tool support). Because the aspects are injected “invisibly” you might encounter unexpected runtime behaviour, especially if the aspects affect each other. Likewise, you have to be careful where the different aspects are advised to avoid some unintended application of concerns. Keep an eye on your pointcuts!

Also, AOP is generally not a first class citizen but an “artificial” extension to existing software environments the woven code comes with an overhead, especially with runtime weavers. Most business applications are I/O bounded, so whether a method call takes 1 microsecond or 1 millisecond does not matter much if the method itself takes 50 milliseconds to run. Just be aware that advising a join point in a performance critical loop might be something to reconsider.

Therefore – as with any tool – use AOP wisely. Use it when it makes sense, just don’t go crazy with your new toy.

Wrapping it up

This concludes my introduction to AOP. I haven’t covered all details of the topic (like inter type declarations or advice precedence). Still I’ve hopefully shown the need for and benefits of AOP well enough to have awaken your interest in the next parts of this series. Feel free to leave a comment for questions and suggestions.

In the next part of the series I’ll how we can use Castle DynamicProxy as a starting point for a AOP-esque separatio of concerns.

Other articles in this series:

Posted in: Tools | Other

Tags: ,

NDepend update brings 64 bit compatibility

September 23, 2008 at 9:00 PMAndre Loker

Patrick Smacchia kept his word: the current version of NDepend added support for dependency graphs under 64 bit operating systems. If you remember, this was one of the few points of criticism I mentioned in my little review on NDepend. Visual NDepend can now visualize the dependencies between types, namespaces and members. If you're hovering over a cell in the Matrix view, NDepend will tell you how the types/namespaces/members on the two axes depend on each other in the small Info window in the lower left area of the screen:

image

When you click the cell a window appears with one or several graphs showing those dependencies visually:

image

While this dependency graph does not show you something completely new, it still a beneficial feature: visualizing the dependencies between types and members makes it much easier to understand them and to communicate them to other developers.

The NDepend people have done their homework. The The dependency graph - like almost every other part of Visual NDepend - uses visual variables effectively to get the maximum information out of the visualization: box sizes, line thickness * and colour are used to not only tell provide the user with additional information on the role and importance of the involved members (lines of code, number of incoming and outgoing dependencies etc.).

All in all, with the 64 bit support for dependency graphs there's one more reason to recommend NDepend.

* There's a drop down that can be used to define what the thickness of edges should represent. As Patrick Smacchia explains in the comments, changing the value does not necessarily lead to a visible change depending on the code that is analysed.

Posted in: Tools

Tags: ,

Chrome didn't survive long...

September 3, 2008 at 9:31 PMAndre Loker

... on my computer. Less than 24 h after I downloaded Google's first own browser I already uninstalled it. Sure, it's slick and lightning fast. But I can't help it: I don't trust it.

Excerpt from the "Google Chrome Privacy Notice":

Your copy of Google Chrome includes one or more unique application numbers. These numbers and information about your installation of the browser (e.g., version number, language) will be sent to Google when you first install and use it and when Google Chrome automatically checks for updates.  If you choose to send usage statistics and crash reports to Google, the browser will send us this information along with a unique application number as well.  Crash reports can contain information from files, applications and services that were running at the time of a malfunction.

It's not completely new that software has a unique ID which is frequently sent back "home". Firefox also has a unique ID used for the auto update feature (see Firefox Privacy Policy, search for "Automated Update Service"). It's the fact that everything you search for, everything you enter into the "Omnibox" is sent to Google (at least in the default configuration if Google is your search engine). It does not take much to come up with some really scary scenarios in which your ID is combined with your search profile. I'm not saying Google is doing it or planning it, I'm just pointing out that with Chrome there's a much higher potential for Google to profile individual usage behaviour.

Still convinced of Chrome? Read on. Excerpt from the "Google Chrome Terms of Service" (Update: this has been changed in the meantime by Google and does not represent the current state of the EULA, see "Update 3" below)

11. Content license from you

11.1 You retain copyright and any other rights you already hold in Content which you submit, post or display on or through, the Services. By submitting, posting or displaying the content you give Google a perpetual, irrevocable, worldwide, royalty-free, and non-exclusive license to reproduce, adapt, modify, translate, publish, publicly perform, publicly display and distribute any Content which you submit, post or display on or through, the Services. This license is for the sole purpose of enabling Google to display, distribute and promote the Services and may be revoked for certain Services as defined in the Additional Terms of those Services.

11.2 You agree that this license includes a right for Google to make such Content available to other companies, organizations or individuals with whom Google has relationships for the provision of syndicated services, and to use such Content in connection with the provision of those services.

11.3 You understand that Google, in performing the required technical steps to provide the Services to our users, may (a) transmit or distribute your Content over various public networks and in various media; and (b) make such changes to your Content as are necessary to conform and adapt that Content to the technical requirements of connecting networks, devices, services or media. You agree that this license shall permit Google to take these actions.

11.4 You confirm and warrant to Google that you have all the rights, power and authority necessary to grant the above license.

I'm no lawyer, but this sounds very much like "Google has the right to do whatever they like with the data you send and receive using their services". This is unacceptable.

Call me a follower, call be paranoid, call be simple-hearted. But I'm not willing to take part in this new chapter of informational transparency. I know that Google is by far not the only threat for privacy. But the more we combine different services of one specific provider the more traceable we become. I'm always open for a discussion, so if you feel like being able to convince me, have a shot at it!

Thanks to Hennie for pointing me to the Chrome terms of service.

Update: "This Post Not Made In Chrome; Google's EULA Sucks" - a nice post from someone who actually is a lawyer.

Update 2: As far as I understand, Chromium (the open source project on which Google Chrome is based) does not fall under the same terms of service, so chances are that if you're using a Chromium build you don't grant any license for submitted content to Google.

Update 3: In the meantime Google has replaced paragraph 11 of the Chrome Eula:

11. Content license from you

11.1 You retain copyright and any other rights you already hold in Content which you submit, post or display on or through, the Services.

Posted in: Tools

Tags: ,