MethodLogger - Hook into method calls in .NET binaries

Senthil Kumar just posted a gem of a tool on CodeProject. MethodLogger is a simple command line application that changes running IL code to hook into methods and do something when they start/end (essentially for logging purposes obviously). Looks pretty cool to me, as well as useful - he's got my 5 points vote!

Could the ESP Game replace CAPTCHA ???

A few days ago I talked about the Human Computation and the ESP Game. I haven't stopped thinking about it since...

What I was thinking was - why not leverage the principles of the ESP Game to replace CAPTCHA, and tranforming this "annoying, inproductive, necessary" thing (CAPTACHA) into something that is "annoying, productive, neccessary"?

Let me explain:
Image you are creating an account in GMail. At some point, instead of being given a CAPTCHA, you are showed an image of a little boy dressed-up like a cow-boy with a huge mustache. And instead of having to type these annoying skewed letters, you must type a word that represents what you see in the image (maybe with a "taboo" list that says that the word "BOY" cannot be used). Since "BOY" is already taken, you type "MUSTACHE" and your subscription goes on.
What happened in the background is that the system already had a set of potential tags for this image, that is, tags given by one user and still require acknowledgement by another (independant) user. Since the word MUSTACHE was one of these potential tags, the systems can deduce that there is indeed a person behind the computer and lets you continue.

How do you get this initial list? Why do you need it anyway? What happens if the word entered does not appear in the potential tags?

You need an initial list for two reasons:

1. Without such a list you would have to play the real ESP Game - in which case you need to compare the words entered by two users. This could lead to too slow response if one of the "players" has entered his word and the second one is still thinking. You could solve this by having multiple players with the same image, but then it's becoming very similar to my idea of an initial list.
2. You must make sure spammers won't take advantage of your system. With the ESP Game, they could launch thousands of simultaneous request, making them all enter the exact same word - they would stand large chance of being coupled together and provide you with really bad tags (not to mention that the whole idea of avoiding spam would be lost).

To generate this initial list, you need to actually show the user 2 images. One of them is based on an existing initial list (i.e. and image for which you already have a large number of potential tags). The second image has no potential tags at all - it's there to build an initial list for the future. The user will "pass" only based on what he entered for the first image - the second one will be judged in the future, once it's used as an initial list.

With this approach (2 images, one compared to an existing list and the other to build a future list), spammers can't fool you. If you leave enough time between the time you generated an initial list and the time you use the image for actual tagging, you can track words recurring very often for various different images, which could potentially be due to spammers. These tags will be removed and not used as potential tags. Thus, the spammers won't be able to fool you! Also, it avoids any delay and none of the users depends on other users.

Even if you collect hundreds of potential tags for each image, you could still get to a situation that the user entered a word that does not exist in the potential tags list. In this case, you could show him another image. Yet, if you want to avoid annoying the users too much, you can simply show him a CAPTCHA. The result being that each user is given exactly 2 or 3 images (in the latter case the third image is a CAPTCHA) and thus there is a concrete limit to the level of annoyance to the user. Of course, you can be really nice and provide the user a choice between a CAPTCHA and an image to tag.

Finally, this system could provide you a huge amount of image tags very fast. The advantage is that the taggers are very diverse, and the amount of taggers is much more than you would have in the case of a game.

Human Computation

 

Luis von Ahn is a GENIUS!!!

He developed a way to make people label images, mark regions to locate objects in images and is working on more stuff - by making people doing it for him. And they do it for free. And they enjoy it. And they continue to come for more....

For a detailed description, I urge you to go and see his presentation. Alternatively go directly to the ESP Game or Peekaboom (or check out their search engine - Peekasearch).

ESP Game is a game where two people have to label given images. Once they both give the same label for an image, they get scores. The result is a pretty accurate set of labels for the images.

Peekaboom - gives the first user an image and a label. He then must point to areas of the image that represent the label. The second player is shown only the area around the points marked by the first player, and must guess the label based only on the portion of the image that he can see. The result is that the labels are given pretty accurate locations inside the images!

The basic idea of transforming difficult tasks, such as computer vision problems and the like, into games is what he calls Human Computation, and its potential is huge! Porn sites use it to resolve CAPTCHA in a very straight-forward way. The gaming approach is essentially the same, except that it's more moral, it's fun, and could be adapted to a lot of other very difficult problems.

Brilliant!

Waiting for threads

I know I'm stating the obvious, but I've seen this in one too many occasions - I had to act...

Sometimes you have an application that must do some job with multiple threads, and once they all complete, you can proceed with a concluding action. The straight-forward code would be as such:

public class StraightForward
{
    private int numberOfThreads;
    private Thread[] threads;

    public StraightForward(int numberOfThreads)
    {
        this.numberOfThreads = numberOfThreads;
    }
    
    public void DoWork()
    {
        // Launch the threads to do the work
        threads = new Thread[numberOfThreads];
        for (int i = 0; i < numberOfThreads; i++)
        {
            threads[i] = new Thread(new ThreadStart(doWorkImpl));
            threads[i].Start();
        }
        
        // Wait for all the threads to complete
        for (int i = 0; i < numberOfThreads; i++)
        {
            threads[i].Join();
        }
        
        // Do the concluding work
        onWorkCompleted();
    }
    
    private void doWorkImpl()
    {
        // Do something here
    }
    
    private void onWorkCompleted()
    {
        // Do some concluding work here
    }
}

Using a threadpool instead of launching the threads yourself is an option, but then I would strongly recommend Ami Bar's SmartThreadPool. Also, to be able to "join" threads from the .NET's threadpool you may want to use Stephen Toub's ThreadPoolWait. Anyway, for our discussion we will assume you have very good reasons for not using a threadpool at all.

The problem with the code above is that it requires an additional thread for "administrative" purposes only. That is, either the calling thread (like the code above) or an additional thread (to run DoWork asynchronously) must block there, waiting for all the others to complete their job and then execute onWorkCompleted.

The solution is very simple - keep track of the number of threads still running using an integer (that is handled only through the Interlocked class). The last thread to complete will know it's the last one and will be responsible to execute the concluding onWorkCompleted method. Here is a sample:

public class NoRedundantThread
{
    private int numberOfThreads;
    private Thread[] threads;
    private int runningThreads;

    public NoRedundantThread(int numberOfThreads)
    {
        this.numberOfThreads = numberOfThreads;
    }
    
    public void DoWork()
    {
        // Launch the threads to do the work        
        threads = new Thread[numberOfThreads];
        runningThreads = numberOfThreads;
        for (int i = 0; i < numberOfThreads; i++)
        {
            threads[i] = new Thread(new ThreadStart(doWorkImpl));
            threads[i].Start();
        }        
    }
    
    private void doWorkImpl()
    {
        // Do something here
        // ...
        
        // If we're the last thread - run the concluding method
        if (0 == Interlocked.Decrement(ref runningThreads))
        {
            onWorkCompleted();
        }
    }
    
    private void onWorkCompleted()
    {
        // Do some concluding work here
    }
}

Now the code is asynchronous, without the need for a thread that just sits there, waiting for the others to do the work :-)

PRNGs for .NET

I'm still looking for a good commerical numerical library in .NET. In the mean time, here's what looks like an excellent PRNG and Distributions library!

Remember - for cryptographic purposes, pseudo-random number generators are NOT good enough and you should always use entropy-based RNGs such as can be found under System.Security!!!

Leave me out of your MLM scams!!!

MLM (Multi-Level-Marketing) has been really flooding the country for the past couple of years. Every few weeks another friend or relative tries to convince me to get with him into this stupid thing (internet phone, vitamins, water filters, games, whatever... - even a magic potion that supposedly solves any physical problem you may have!!!). They are being filled their heads with a vision of mountains of gold and loose their common sense on the way! I know tens of people who got into things like that, but NONE has EVER made enough money to cover for the time and initial investment demanded from them.

IMO, there are 3 ways to make money:

1. Work, hard, preferably at a job with high demand.
2. Have a lot of initial money and let it do the job for you.
3. Steal.

(In case you were wandering - only the first way is an option for me :-)

Also, when I buy a product, I want to pay the money, get the product, and get on with my life. I don't want to have to do some additional work to make it worthwile having bought the product in the first place. I also don't want my decision of whether or not to buy a product to be dictated by the money it may or not give me at the end of the road.

That's it, I had to get this off my chest...

Binding Enum values to a combobox

Being mostly a back-end programmer, every time I find myself working on a Windows Form I have to remind myself of all these tiny tricks. This time I'll post it on my own blog, so it will be easier for me to look it up in the future.
Say you have an enumerator (MyEnum) and you want to bind its values to a combobox (myCombo) - the simplest way is as so:

myCombo.DataSource = Enum.GetValues(typeof(MyEnum));

Rich Lentil Soup

It's getting cold again, and Winter is getting close - so I'm making soups again*... 
Today I'd like to introduce you to one of my favorites, especially when it's really cold and wet - meet my Rich Lentil (and Meat) Soup:

Ingredients:

• 2 Onions in thin slices/stripes
• 2 cups of green Lentils
• 1 Potato
• 1 peeled** frech Tomato in little cubes
• 3 sliced cloves of Garlic
• 1 Carrot in slices
• 1 small piece of Pumpkin in small cubes (could be replaced by a small sweet potato, but then the soup will be sweeter)
• 2 Celery branches with leaves
• 1 piece of Bone with marrow
• 300 gr. meat (cow or mutton). You could replace this by 1 or 2 additional bones instead
• 5 bay-tree ("dafna") leaves
• olive oil
• salt
• 1-2 tea-spoons of Hawayedge***
• (optional) hot pepper
• (optional) Chili pepper

Directions:

• Fry the onions and the carrot in the olive oil until the onion gets yellowish
• Add the garlic and keep frying for a couple more minutes
• Add all the ingredients except for the spices. Cover with water. The lentils will absorb some of the water, so make sure there is enough water.
• Once boiling temperature is achieved, leave it at boiling temperatures for about 2-3 hours. The water should become completely brown and the lentils should start to decompose. Also, most of the tomato's acidity will be gone by then.
• Now add the spices (salt, Hawayedge, hot pepper, Chili pepper), stirr well, wait a couple more minutes for the spices to catch.
• Enjoy!

Notes:

• Of course, you can play with the ingredients. Today, for instance, I used a sweet potato instead of pumpkin and didn't have celery. I also added one zuccini I had in the fridge. It was a huge success.
• The tomato is there to add some acidity. Some people like it even more acid, and use lemon - I prefer it with 1 tomato.
• The thing I like with the pumpkin is that during the cooking it completely desintegrates, making the soup thicker.
• In many cases I find myself taking one large cup of this soup to replace a whole meal (maybe with some bread on the side).

* If you're interested in more of my soups, check out my:
Chicken Bouillon
French Onion Soup
Orange (color) Soup

** To peel a frech tomato make an X shaped cut at it's bottom, and put it in a container with very hot water for a few minutes. You will then be able to peel it effortlessly.

*** Hawayedge is an oriental coctail of spices (originally from Yemen) made of black pepper, cumin, cardamom ("hel"), turmeric ("curcum") and coriander ("cusbara"). Maybe with other things, I wouldn't know - I buy it already made from my spice-man...

Numerical Library in C#

I'm looking for a good numerical library in C#. So far I've found the following:

1. Extreme Optimization - Looks nice, documentation a little weak. Seems rather expensive for the amount of functionality it supplies. Also, source code is only available for large licenses (with an extra payment).
2. Visual Numerics IMSL - From references I found on the internet, it seems to be the most popular, and certainly the longest on the market (the company is 35+ years old). They also propose a very large range of functions. But they seem to only support .NET 1.1 !!! And what's more, their last version was released only last June, so it's not that they are about to release any .NET 2.0 version anytime soon. Besides the fact that I work only with .NET 2.0, I really don't tend to like products from companies that are so slow they can't follow the market on time.
3. NMath - Looks very promising. They also provide a means of purchasing only packages with what you need, and give you full source code without additional charge. I'm just curious about the company, who'se been around for only 4 years or so (compared to Visual Numerics).

Any suggestions?

Skype Gaming Infrastructure - Coooool

Take a look at this: http://www.codeproject.com/csharp/skypegameinfra.asp

Michael Gopshtein shows you how Skype can easily be used to build an online game (or any other multiple users online application for that matters) - no dedicted server attached! And, to make it even nicer, he wraps it all with a nice C# library.

Awesome!

"The District" - WTF

I was taking a break, eating a salad in front of the TV, watching half an episode of "The District". There was a whole fuss going on about a kidnapped little girl. The parents got the ransom email, the ultra-smart police-guys analyzing everything on the spot, etc.
Suddenly one of the geeks says: "We've got a partial trace - it's been sent from an anonymous server using a 256-bit key". Then the father's associate jumps saying: "256... Hey, that's the software we've developed!".
And then of course they concluded that the kidnaper must be one of their past employees... (I didn't see whether they were right - it's commercial time now)

Need I say more?

Networked-RNG

True Random Number Generators are hard and almost impossible to create. Most computer programs use pseudo-Random Number Generators (PRNG) which are, in fact, not random at all. They simply use a mathematical function (most commonly LCG) with a seed based on the system's clock.
For security applications, these simple PRNGs are bad bad bad - they create a giant hole in the security mechanism through which hackers can fairly easily penetrate. So entropy-based RNGs are usually used for cryptographic purposes (for example Window's CryptGenRandom). These RNGs try to collect randomness (actually it's entropy) from as many resources as possible: keyboard events, mouse events, hard-disk events, network events, etc.
Linux also has such an entropy-based RNG, but it has been shown to be weak in some circumstances.
To my knowledge, Windows' counterpart (CryptGenRandom) hasn't yet experienced such a faith...
BTW - an important problem with the cryptographically strong RNGs on a computer is that sometimes they must wait for enough entropy to occur before being able to returning a new random number, causing the application to stall (and making it difficult to generate a large amount of random numbers at short intervals).
Would it be possible to have a really random RNG?
Well, there are various physical phenomena (such as some nuclear processes) that could be used, but it's complicated to incorporate this into every PC :-)
An idea came to my mind - Imagine we had a web service, generating simple random numbers, based on previous calls to the service (and maybe a few other entropy sources). Then each user would get a random number that is influenced by some other user, completely unknown to her. The main idea behind this is that the randomness is aquired from the various independent requests, making it essentially random.
A simulation of such a generator could be easily created by using Blogspot's "Next Blog" link, where you are randomly forwarded to another blog. Take the link you got (or some information inside the blog), hash it somehow (MD5 hasn't been broken yet), and what you get is really random (I checked - a blog doesn't refer twice to the same blog).
This is all very fuzzy, and I'm sure it's flawed in many ways. Yet I think that the principle could be interesting - using user navigation information to provide an RNG service over the web.
And what about performance - having to execute a web request to get one random number isn't very nice!? Well, I didn't say it's perfect, did I? Yet it may (maybe) be extended to return pools of random numbers, thus requiring much less web calls. Also, if you're already running a web site, you may be able to use your own users' information to generate a local RNG only for yourself.

OK, enough babling, ciao!

Afraid of moving to IE7

I know that using IE is not very "geeky", but I don't care - I'm an IE user. I've tried FireFox several times (including the latest version 2), and even installed the latest Opera browser. Both had real difficulties coping with my banks' sites (Opera remained blank and FireFox often gets completely stuck, up to getting my whole machine stuck!). I want to use one single browser  - without needing to remember which browser works best with which site. I know that the culprits are the web developers and not the browsers (probably), but I don't care -the bottom line is that I can't use either of them for all my browsing tasks.
Conclusion - I have to keep using IE.
Now IE7 is out, and it looks pretty promising - enhanced security, tabs (finally), RSS, etc. Not as rich as FireFox or Opera, but still - much better than IE6. The problem is that I don't know whether the sites I usually visit properly support IE7, and I really don't feel like counting on uninstalling it in case of problems.
Dillema....

Things I'm missing in C#/CLR

Although C# is a great language, I'm still missing some features. I know it's mostly a matter of CLR limitations, but I'm missing it nonetheless.

1. Signature-free Delegate - imagine you have a system that works intensively with configuration/settings files, loading stuff at runtime. Now you want this system to be able to dynamically receive a delegate from one source and parameters from another source, and run that delegate with those parameters. At compile time, the only thing you know is that you have to run some delegate - you don't know its signature. I'd like a way to define a delegate that accepts any signature, and a technique to dynamically call it with whatever parameters I get. Of course it means that important stuff cannot be checked at compile time, but sometimes it's the best way to provide a generic solution (if you're interested, I could post an example of what I mean in some later post).
2. Multiple Inheritance - I know, this is almost a theological issue (much like the Good/Bad Agile debate going on lately). I also admit that it doesn't happen a lot that I really need multiple inheritance. Yet, sometimes I do, and at these times, I'm really p'd off for not having the option (and don't give me the brainwashed crap feeded by Microsoft that you never need it and can always use some pattern to work around it!). BTW - Eiffel.NET has full multiple inheritance support.
3. Method Signatures - as in C++, methods can't differ only by return type. That sucks, plain and simple.
4. Multiple Return Parameters - who said a method must return only one parameter? If I want to divide two numbers and get both the integer result and the remainder - why do I need to get one in the return value and the other as an "out" argument?
5. Interactive Scripting - I've written enough about that.

.NET 2.0 Tip: Strongly Typing Configuration Settings

Every body say Thanks to Sahil Malik for his cool tip!

An Amazon WTF

(From Amazon.com)

"Monte Carlo and Quasi-Monte Carlo Methods 2004 by Harald Niederreiter and Denis Talay (Paperback - Dec 31, 1899)"

Unless the writers were able to travel in time, I really don't see how they could publish a book in 1899 about stuff they worked on in the 21st century...

P.S: I would have posted an image, but it's such a pain with Blogger that I try to avoid it as much as possible :-)

Joke of the day

A lawyer died and arrives at the gates of heaven. He immediately starts making trouble, claiming he wasn't supposed to die. Eventually, he manages to speak to one of the angels in charge and says: "I don't understand what happened. I was happily sitting in my office, working on a very important case. Then, out of the blue, I just died". The angel wasn't very impressed, opened his laptop, searched for the lawyer's file and read in a monotonic tone: "I'm sorry mister, it says here that you died of old age". "But I'm only 35 years old!!!" -replied the crying lawyer.
"Well, that may be right, but if you count the hours you've charged your customers, you've reached the incredible age of 187..."

Scripting in C# - take 3

I've been ranting about the lack of scripting capability with C# (here and here).

Another example of what I meant could be M# (via Larkware) and F# - both having an interactive interface.

Why, oh why is there no such thing for plain old C#?

[UPDATE] After inquiring with Extreme Optimization, it turns out that I misunderstood their site. They DON'T have an interactive scripting capability (yet), so only F# has it, not M#...

Could we benefit from two mices?

We've been using a mouse and a keyboard for ages now. It's an axiom with contemporary computers: computer, monitor, keyboard, mouse.

Did you ever consider using two pointing devices instead of one? With one mouse, you can manipulate the mouse and keep one hand on the keyboard. But in most cases you're so used to typing with two hands, that you're mostly incapable of using your keyboard with one hand anyway.

I'm used to using my two hands simultaneously, sometimes doing different things each. Could there be a way to put that in action with two pointing devices?

Scripting in C# - take 2

Yesterday I complained that I want a way to run scripts in C#. Funnily, Leon Bambrick (The Secret Geek) published an add-in to VS (for VB) that does part of what I'm searching for approximately at the same time. This add-in lets you mark some VB code and execute it on-the-fly. That's great, and it's part of what I am looking for (although his add-in is for VB only, and I'm looking for a solution for C#). What I want in addition to that, is a command-like UI where every expression you type gets executed once it's completely written.

Here's an example of a scripting session as I perceive it (Matlab users will probably feel at home):

>> int number = 5;
ans:
     number = 5
>> for (int i=0; i < number; i++)
     {
         Console.WriteLine(i.ToString());
     }
ans:
     0
     1
     2
     3
     4
>> $Clear(number);
ans:
     number cleared from memory
>> class MyClass
     {
          MyMethod(string input) 
          {
                    Console.WriteLine(input);
          }
     }
ans:
     MyClass declared successfully
>> MyClass myObject = new MyClass();
ans:
     myObject created successfully
>> myObject.MyMethod(1234);
ans:
     Syntax error - MyClass.MyMethod(string input) does not accept input (1234)
>> myObject.MyMethod("Hello World!");
ans:
     Hello World!

A few notes:

1. As you can see above, expressions can span more than one single line. The engine simply waits for the expression to be completed (semi-colon or brackets). So the for loop or the class declaration are being executed only when the closing bracket is typed.
2. Although I wouldn't use this for very large and complicated classes, there should be support for classes and any other construct of the language.
3. Each expression ends with a feedback from the engine about its execution (the "ans" regions - taken from Matlab)
4. Errors don't throw exeptions, but rather give you as meaningful an explanation as possible
5. Since the engine must keep some data in memory, we need to be able to free it at will. That's what the $Clear command would do. I suppose we might need support for a few more such scripting commands (the less the better).
6. Once you have such a scripting engine, evaluating portions of some code (like Leon's add-in) becomes trivial.

P.S: For all those who say - "Dude, with Powershell you can do that and much more", I say: "Dude, as long as it's in another language it's not the same. It means I can't take it for granted any C# programmer will know the scripting language as well as he knows C#. It also means I can't easily copy code from my scripting console to my code - I have to first translate it into C#. So no - unfortunately Powershel is still not there in terms of ease-of-use."
Still about Powershell - I have nothing against Powershell. Actually, it was about time decent scripting became available for Windows users (and sure enough Powershell is way beyond decent!!!). What I don't understand, though, is why was there a need for a new language? Wouldn't it have been nicer to have an additional set of libraries to support the various functionalities available in Powershell (WMI, IIS, File System, etc.) - add these libraries to the .NET Framework and add a scripting engine as described above? Everyone would be able to pick his favorite language and use it for scripting. Granted, Powershell provides some functionalities with just a single command, that would have required much more lines of code in any other language. But I think that's not a good excuse - you can always wrap those commonly-used functionalities inside some static function, providing the same final result (single function call to do a rather complicated task).