SQL Server 2005 - Frustrated by a good feature

I guess that most people who have been using SQL 2005 for some time already know about this. I, for my part, have worked a lot with SQL 2000 in the past, but never had the chance to really work with SQL 2005 until recently. Now I needed to connect to a server on a remote machine and had this frustrating experience...

1. Scenario: trying to connect to a SQL 2005 server from a remote machine fails. It gives the following message: "Sqlcmd: Error: Microsoft SQL Native Client: An error has occurred while establishing a connection to the server. When connecting to SQL Server 2005, this failure may be caused by the fact that under the default settings SQL Server does not allow remote connections."
2. After getting this message once or twice you take the time to actually *read* it, and start checking the settings for the SQL Server.
3. Like anyone with experience with SQL 2000 would do, I opened the SQL Server Management Studio. There I looked at the properties for the server instance and saw that the configuration seems fine and the server is configured to accept remote connections.
4. Back to square one. From this point I started looking for the culprit...
a. Maybe it's the FireWall? I configured the FW to trust my local network, but maybe it fucked up somehow? Disabling the FW quickly showed me that's not the problem.
b. Maybe there is some problem with the SQL version and I should upgrade? Seemed unlikely, yet I checked it out. Turned out to be irrelevant since I was already using the latest version (SP1).
c. Maybe I did something wrong with the connection string I used? Tried all possible variations - nothing worked...
5. In the back of my mind I started to rethink about this error message I got. It says that by default, SQL 2005 is configured not to allow remote connections. I don't remember having changed that - so how come it's configured to accept remote connections? Could there possibly be some other configuration parameter that has some impact on remote connections?
6. I went over all the configurable parameters for the server instance and for the database (from SQL Server Management Studio) - nada
7. Well then, I'm on the verge of throwing my computer out of the window. I'll give Google a last try. Then I found this: http://support.microsoft.com/?kbid=914277&SD=tech
8. It turns out that there is a much more elaborated way of configuring remote connections in SQL 2005. This is done through the "SQL Server 2005 Surface Area Configuration". It is, of course, a good thing they have added this wealth of configuration options - but why couldn't it be accessible from the Management Studio? Couldn't they add an "Advanced" button on the remote server connections options that opens this Surface Area Configuration??? And if they didn't want to put too much things in the Management Studio - why did they give an option to configure remote connections there, when it can't work on its own anyway???

If you ever run into something similar - please remember this, it will save you some valuable time ...

Safely firing events in C# without locking

Long time no see…
I’ve been busy with plenty of stuff, and I don’t think this will change much in the next coming months – I’m trying to finish my thesis. So basically I put my blogging into (very) deep freezing state. I don’t promise it won’t remain that way until I finish with my thesis…
Anyway, yesterday I went to hear Juval Lowy at the C++/VB Users Group meeting. As usual, his talk was excellent and very informative. Also as usual, I learned as much from things he said on the side as I learned from the actual presentation. One of these side-things I learned really stunned me. It’s the simplicity in itself, but I never saw anything written on this, and even when I tried searching a bit about this on the Internet, I couldn’t find any concrete discussion.
Problem definition: How to properly handle events in a multi-threaded environment?
Just to get in sync, look at the following simple event-handling code:
public delegate void DummyEvent();
public class EventDemo
{
…
private event DummyEvent dummyEventRace;

public virtual event DummyEvent DummyEventRace
{
add
{
if (dummyEventRace != null)
dummyEventRace += value;
}
remove
{
if (dummyEventRace != null)
dummyEventRace -= value;
}
}

protected virtual void onDummyEventRace()
{
if (dummyEventRace != null)
dummyEventRace();
}
}

In a single-threaded world, this works fine. I make sure every access to the event first checks that it’s not null, and thus avoid the very annoying exception, and life is pretty cool.
However, in a multi-threaded environment, you could easily get into a race condition:
Thread A wants to fire an event and calls onDummyEventRace() . I manages to check whether dummyEventRace != null and then comes a context-switch.
Thread B gets the context and tries to remove an event handler – it fully removes it from dummyEventRace and since it’s the last handler, dummyEventRace becomes null .
Thread A gets the context again and – Ooops – exception!!!

So, to avoid this kind of problems, what most people would do (me inclusive) was to add a synchronization mechanism, that ensures each of the operations with dummyEventRace would be done atomically. It would look something like this:

public delegate void DummyEvent();
public class EventDemo
{
…
private object eventLocker = new object();
private event DummyEvent dummyEventLock;

public virtual event DummyEvent DummyEventLock
{
add
{
lock (eventLocker)
{
if (dummyEventLock != null)
dummyEventLock += value;
}
}
remove
{
lock (eventLocker)
{
if (dummyEventLock != null)
dummyEventLock -= value;
}
}
}

public virtual void onDummyEventLock()
{
if (dummyEventLock != null)
dummyEventLock();
}
}
This solution solves the race condition issue completely, but introduces a potential dead-lock issue. If the handler of the event must get hold of another resource inside its event handling method, and the same resource is being locked by the thread that tries to remove the event… In other words – it works, but could get nasty.
Anyway, until yesterday, not only did I think that was the proper way to do it – I thought it was the only way to do it.
I was WRONG!!!

Take a look at this:
public delegate void DummyEvent();
public class EventDemo
{
…
private event DummyEvent dummyEventNoLock = delegate {};
public virtual event DummyEvent DummyEventNoLock
{
add
{
dummyEventNoLock += value;
}
remove
{
dummyEventLock -= value;
}
}

public virtual void onDummyEventNoLock()
{
dummyEventNoLock();
}
}
Amazing, isn’t it? The dummyEventLock never turns back to null so there is no reason to check. Thus, no need to lock and you’re rid of both the race condition problem and the dead lock risk!!!
How come this is nowhere to be found in the MSDN documentation (to my knowledge) ???

Matlab, Memory Leaks, Dual-Core CPU and what's in between

[UPDATE: If you're searching for ways to better use your multiple cores, the R2007a release seems to support multithreaded computations. More on this on my recent post.]

I haven’t posted much recently. Well, I can say for my defense that I was sick (actually we were all three sick, each in turn) and shortly after that we went on vacation. Also, next week I have reserve duty, so I’m not going to post much in the near future either. I have still a pending SQL post I talked about a long time ago that is waiting for me to finish the last quarter of it. Shame on me…

Anyway, I wanted to share some performance problems I had to solve recently. They are all related to Matlab, so if you’re not into Matlab I guess it will be of little interest to you J

I recently purchased a very powerful machine with a dual-core CPU, 2GB DDR2 memory, etc. I’m using it to run some very extensive calculations, mostly in Matlab. However, I got very frustrated because:

1. After a few hours of intensive execution, Matlab throws me an “out of memory” exception. And, sure enough, it’s taking up more than 2GB of memory (note that I have 2GB ram + as much cache as I like).

2. I just couldn’t make the PC use all of the CPU. I didn’t expect it to use both CPU’s for a single Matlab process. I expected around 100% usage on one CPU (core actually) and close to 0% on the second. Instead, sometimes I got around 50% on both and most of the time it was around 20-30% for both CPU’s. So I was barely using ¼ of my processing capability (and only ½ of what I expected).

To tackle the first problem I tried everything, read a whole bunch of web pages but in the end I didn’t get anywhere. My first intuition was that for some reason Matlab is not releasing memory it has allocated, probably due to some error of mine, and I just couldn’t find the source of the problem. After some time I started to think that maybe the problem is with the SVM library I am using repeatedly in my process. As I mentioned in a previous post, I have started using a Matlab wrapper over SVMLight. After digging into both libraries, adding some code to track memory allocations, I managed to prove that both libraries don’t free all the memory that they allocate. While working on all this, I learned a bit about MEX-Files (C functions that can be called from Matlab), since the wrapper library is a MEX, of course. So I learned that there are various ways to allocate/free memory when you’re working with MEX-Files. One, of course, is the C malloc/free functions. Every memory block allocated with “malloc” must be freed (other wise you’re leaking), and if you’re using Windows, you must free it exactly once (more causes exceptions). Another problem with malloc/free in MEF-Files is that memory allocated by “malloc” should not be returned to Matlab. An alternative to malloc/free is using Matlab’s mxMalloc/mxFree pair. Memory allocated by mxMalloc can be returned to Matlab. Additionally, when the MEX-File is released (i.e. the function call is over), any memory allocated with mxMalloc that was not part of the returned variables is automatically released.

So, all I needed to do was make sure that BOTH library would perform ALL their memory allocations with mxMalloc, and then I wouldn’t need to take care of the freeing – Matlab would do it for me (as a side-note I must tell you that in my scripts, I call these libraries over and over again in a loop, so there are many short calls to the MEX-File). So I simply aliased malloc to mxMalloc and free to mxFree (remember the good old “#define malloc mxMalloc” ?) and voila – problem solved (“worked around” would be a better definition). I left my process running for 8 hours, and the memory used by Matlab increased only by a few small MBs! Yipee!!!

Once I had my memory problem resolved, I could start thinking about the CPU. So I changed my script files to separate the workload into 2 processes, instantiated 2 instances of Matlab – each running part of the job. At that point, the most incredible thing happened – once I started running the second process, the CPU turned flat (< 5% on both cores) and both processes got stuck with almost no progress at all!!! This drove me totally crazy! I tried changing the process affinity (i.e. define each process to run on a separate CPU), changed some system performance settings and even installed a hotfix that deals with performance problems on dual-core machine (you can download it from links in this thread) – but nada! Then I thought of trying something totally different. The code I was running was located on a remote computer. Actually, it was on a Disk-on-Key that was connected to my laptop (and the new PC has a network map to the DoK). I’m using this configuration because I take the DoK often with me and need to work on it from my laptop. So I connected the DoK directly to my workstation (the new PC with the dual-core CPU) and voila – both processes work marvelously, taking up 100% of both CPUs (cores)!!!

What have I learned from all this:

1. As always, the source of the problems is usually where it makes most sense. With the memory, the problem is with 3^rd party (non-commercial) code. This code was written for research purposes and I guess that as such there was little effort done to ensure no memory leaks. With the CPU – the problem with the issue of working on network mapped disks. I’m sure that once you’ve read everything above you’ve told yourself – “hey, but it’s so obvious!”. So as always, I should have followed my instincts in the first place…
2. My little aliasing trick with malloc-mxMalloc is something to keep in mind. It might not always be a good idea (especially for long-running code) because if there is a leak it remains until the MEX-File is unloaded, but I guess that in many cases it’s a really good and simple workaround with minimum risk to harm code you’re not familiar with.
3. Working from a network drive involves some hidden constraints that must be further investigated. It may be something that is specific to Matlab. Don’t know yet.

SVM Library in Matlab

For the past few weeks I’ve been using SVM as part of my research project. Until recently I used the SVM functions available in Piotr Dollar’s Image and Video Toolbox for Matlab. I am, however, constantly looking for alternatives. From what I saw on the Internet, it seems that one quite popular SVM implementation is SVM-Light by Thorsten Joachims. Add to it Tom Brigg’s MATLAB/MEX Interface to SVM-Light and you can run it from Matlab. The only thing you need is to put the Patch.exe utility in a folder that is in your system path and follow the installation instructions.

I’m still new with SVM-Light, so I don’t know exactly what all the parameters mean and how to tune them. A-priori it seems to work slightly better in terms of generalization accuracy and also run a little faster (with mostly default parameters and the same kernel function and parameters as Piotr Dollar’s implementation).

If you know of some other interesting SVM implementation, especially one that is available in Matlab, please drop me a line.

Windows XP Tweaks

While installing a new computer, I was trying to remember how to increase the number of simultaneous connections possible in Internet Explorer. So I stumbled on this site, which has an enormous amount of tweaks. Many are not interesting (to me) or already handled by PowerToy’s TweakUI, but I did find several new tweaks I really liked. Specifically I liked (and used) the one about how to add "Copy To Folder" and "Move To Folder" to Explorer context menu.

Another relevant site, with much much more options (although many are too basic to be interesting) can be found here.

I don't want the cellular companies to fry my brain!!!

Since the cellular companies have introduced their 3G solutions, the number of cellular antennae is increasing exponentially. Today I decided to do a little test. I counted the number of antennae on the highway between Caesarea interchange and Kfar Shmaryahu (it’s less than 35 km).

I only counted those antennae that are visible from the highway. I guess I’ve missed some, since I had to look at the road once in a while and also more and more antennae are hidden to the casual eye, to avoid protestations of people who live nearby.

I was able to count more than 40 antennae on this small piece of road!!!!!!

Need I say more?!?!?! Where does this stop???? When are there going to be laws that have the citizens' health and well-being in mind?

Just to set things straight - I'm NOT against cellular phones in general. I myself am using a cellular phone on a daily basis and understand its importance in today's life. I don't, however, like the idea of having antennae spread all around me, only to support 3G features, which frankly I feel are mostly superfluous.

Windows XP Performance tweaks

I’ve been trying to do some tweaks to my laptop’s performance. Found some very nice stuff here. The most interesting to me was how to speed up folder access – it makes a real difference with heavy folders.

In the same spirit – I recently got an incredible USB flash drive. It’s the 2GB Lexar JumpDrive Lightning. It’s an amazing piece of technology. It’s wrapped in a stylish stainless steel case, has a built-in support for password-protected secure area and is really lightning fast. I mean REALLY fast!!! It’s so much faster than a good hard-drive – even working in the secured area of the drive (which is where most of my stuff is stored). It’s got only one (rather small) liability, which is that you can only format it to FAT (no FAT32 and certainly no NTFS), which I detest. But the speed, the speed, the speed, the good software that comes with it, the lifetime warranty, the sustainable case and of course the speed covers for this annoyance.

So I was thinking – well, I could spare half a GB on my flash disk and use it for system paging, maybe this could give my OS a boost? Unfortunately, until now I haven’t been able to set this to work. For some reason, when I define a page file on the flash disk (fixed size), everything seems fine - I’m told to reboot for the changes to take effect, but when I do, the page file is not created on the flash disk (although according to the configuration it should be there). Go figure…

Seeking for help with the Google Image Search API

I’ve been asked for help implementing my Google Image Search API into a Mac “widget”. From what I understood, writing “widgets” is more or less like writing web pages (JavaScript, CSS, HTML, …). Also, most of the work is already in place, there is only the need to plug in the actual Image Search code. I guess this can be done by using the some kind of Regular Expression engine for JavaScript with the same regular expression I’m using in my C# code.

Is there anyone willing to help? I don’t have enough Web development experience to do it, or time to learn it.

Drop me a line if you’re interested.

Orange (color) soup

Yeah, well in Hebrew it sounds better, since there is a clear distinction between "orange" the color and "orange" the fruit. This post refers to the former ;-)

This is the soup I made last week-end. It's not the first time I've made it, but for some reason, this time it got a particularly great success. And it's so simple...

Ingredients:

• 3-4 Onions/leeks. I even use both sometimes
• Pumpkin (don’t ask me to give you quantities – I have no idea)
• 4-8 Carrots
• 3-4 Sweet Potatoes
• 3 Sliced Cloves of Garlic
• 1-2 liter of Chicken Bouillon
• Some olive oil (or yellow butter) to fry the onions/leeks
• Salt, Chili powder, Nutmeg

Directions

• Cut the onions (leek) in thin stripes.
• Fry the onions (leek) for at least 1 hour on a VERY LOW fire. They should NOT burn!
• Add the garlic. Stir well until you start feeling the smell of the garlic (just a few seconds - it shouldn’t burn!).
• Add the vegetables (pumpkin, carrots and sweet potatoes) and the bouillon. You may need to add some water to make sure all the vegetables are covered.
• Leave it cooking at boiling temperature for 1-2 hours.
• Once all the vegetables are really tender (almost melted), crush everything with a blender.
• Add the spices according to taste.
• As you must have noticed, the quantities for the ingredients are not very accurate. To get the best substance, the vegetables should be cut into rather small cubes (around 2x2x2 cm) and there should be just enough liquid (bouillon + water) to cover all the vegetables.
• When served, it’s nice to add some fresh Basil.

Windows Live - Microsoft's new search engine

[Update 09-Mar-2006] - The Image Search works. Or to put more accurately doesn't work. Just compare the search for "apple cake" with Google Image Search. I really don't see what they were thinking when they claimed to be a worthy competition...
[Update] - After playing a little more with Windows Live I noticed the following:
1. It seems that MSN Search and Windows Live return the same results for all queries. So my guess it they already started merging them into one.
2. There seems to be a significant improvement in the relevance of the results, as compared to MSN Search (in its previous version).
3. It's still too early for me to give an exact opinion and comparison with Google and Yahoo!

A Beta version of Microsoft's new search engine, Windows Live, is now available.
Like most people, I have a set of standard queries I like to try on new search engines, to which I know what to expect. Most of them are image queries, since this is strongly related to my thesis, but unfortunately they don't support image queries yet.
So I tried a web query I like a lot: "ariel shamir". Ariel Shamir is my thesis supervisor, and I like searching his name for the following reasons:
1. He has quite a few publications and should have some kind of footprint on the Internet.
2. His first name, Ariel, is the same as Ariel Sharon the prime minister, and thus many search engines are too much biased by that.
3. His last name, Shamir, is the same as one of our previous prime ministers, Yitzchak Shamir, which incurs yet another (wrong) bias.
4. He is sometimes referred to by his nickname, Arik, adding another level of confusion.

So all'n'all, it's a simple, but tricky query.
Well, here are some results for this simple search:
Google
Yahoo!
MSN
Windows Live

Some interesting notes about Windows Live's results:
1. Ariel Shamir's actual home page is nowhere to find in the results (at least in the few first tens).
2. Windows Live fell in both pitfalls - the first result is about Ariel Sharon and the second is about Yitzchak Shamir.
3. There are many links to Ariel Shamir's papers and to stuff related to him.
4. Maybe the most annoying of all is the similarity between the results in MSN and in Windows Live. Did they just give it a cool name and an even cooler GUI and leave the engine as is???

To sum up, this is by far not a thorough test, but still a dissapointing one. It leaves me with much less anticipation for the final release of Windows Live than I was before it.

Preventing violent crimes

There has been yet another high-rating violent crime last week-end. Inbal Amram, 20, was killed when she took the car for a few minutes to get her sister. The police has still no clue as to who might have done it nor why.

I’m sick of it! And frankly, I’m truly scared. I have a 2-year old daughter, and if these kinds of things happen so often today, what will it be when she reaches 16?

What drives me crazy even more is that there is a pretty efficient way to prevent this. We all had to endure, during all our school years, these stupid sports lessons. I call them stupid, because for those kids who are much into sports, the lessons were just a game and too easy for them. And for those kids who are really not into sports, it’s boring, tiresome and a waste of time altogether.

Why not use the time, energy and money, in teaching kids self-defense. I’m not talking about a crash-course of 3 lessons. I’m talking about seriously teaching a martial art, throughout all the childrens’ school years, that would teach them how to defend themselves. Make it part of the education program. To be more precise, I would make it obligatory for girls. After all, let’s face it – most violent crimes are performed by men against women.

I know that this is not a very orthodox solution. But I know what you can learn from practicing a martial art for years. You learn discipline, self-control and improve your self-confidence. When the moment comes, you know what to do and are not afraid of doing it. I know that if my girl will start practicing something like that from young age, I will sleep much better at night, when she’ll grow up.

Working on the first SQL For Developers Tips and Tricks post

I've started working on the first post in the series "SQL For Developers Tips and Tricks post". It will cover useful links, things you may not know about Query Analyzer, some guidelines about using SQL Profiler and will show you the 'bcp' utility, which is very helpful to transfer data accross databases.
I hope to post it in the next few days - stay tuned!

Getting information in Matlab

One of the things I really hate with Matlab is that when you don’t know exactly what you’re looking for, you stand a good chance not to find it. Hmm, this doesn’t sound right, let me rephrase: when you don’t know how what you’re looking for is called in Matlab, you stand a good chance not to find it.
Today, for example, I wanted to retrieve the file that implements a specific function. Since I’m using several free toolboxes I downloaded from the internet, I wanted to know whether the function is part of the Matlab installation or of one of the toolboxes. I started by typing:

>> help pca

Which gave me a complete description for the 'pca' function. Yet, it didn’t provide its source. I couldn’t find any allusion to this function in the Help, which indicates there is a good chance that this method comes from an external toolbox. Note however that it’s not always the case – some functions that are part of the installation are not easy to locate in the Help window, even if you know their exact name.
After several minutes of frustration I discovered that the following line:

>> which pca

returns the full path of the file that implements the function ‘pca’ (and indeed it was from an external toolbox).

My frustration was – how the h… am I supposed to know that the function that provides this information is called ‘which’ ??? My situation was even better than most cases, because I vaguely remembered that there is such a function. Sometimes you don’t even know exactly how you can do what you actually want to do, and then looking it out in the Help becomes a real nightmare.

That’s all, just had to get this off my chest…

SQL for Developers Tips and Tricks: 1. Introduction

Lately I’ve been mostly working on research projects related to Machine Learning (both for my thesis and as a private contractor). Therefore, my ability to blog on development issues I’m encountering is somewhat limited. Therefore I have decided to start blogging about stuff I’ve learned and used along the years, both to keep it as a written memory and maybe to help some readers…

So for starter, I’ve decided to summarize various SQL issues I liked/learned/used. Some I got from fellow developers or DBA’s and some I had to learn on my own (sometimes at great personal risk to become completely crazy). I’ll try to do it in an organized fashion, each blog focusing on some other subject. The vantage point is that of a software developer not a DBA. I have seen many cases where a developer coming from C/C++ started using SQL. The first few days she invests in getting a good grip at the basic syntax (SELECT, INSERT, UPDATE, …). Then she starts writing stored procedures and learns some SQL programming concepts (like CURSORs). At some point, she starts getting acquainted with Views and Functions and learns various helpful built-in SQL functions. Then, slowly but surely, the problems start to arise, usually performance. Once the performance issues are more or less under control, there comes a time to design a new database. This new database inevitably includes many flaws, which will have to be dealt with at some point or another.

I have gone through this, and many good programmers I know have gone through a similar sequence. The purpose of this set of posts is to try and give the programmer a set of tools and knowledge base that can help her go through this ordeal with more ease and confidence. This is by no means a complete tutorial, but I still hope it will be of help. If there are any specific requests, please send a comment with the details. If I know the answer, I’ll try to asses it in the future posts.

One last thing before I present the list of issues I intend to cover – since I’m still too short in hands-on experience with SQL2005, most of the information will be provided with SQL2000 in mind. I guess that most of the issues are applicable to SQL2005 as well, although the new SQL version certainly brings a whole series of new issues (with the CLR and all) as well as additional and revised tools. Also, these posts will assume familiarity with the basic SQL syntax (SELECT, INSERT, UPDATE).

Following is the list of issues I plan to asses. This is a preliminary list and may be altered along the way, but should give you some idea of what to expect.

1. Introduction
2. Basic tools you should keep at the ready
1. Helpful web sites
2. SQL Query Analyzer – things too many people learn too late
3. SQL Profiler
3. SQL Programming for Performance – basic programming tips that can help reduce future performance problems
1. Similar code that runs faster
2. Cursors
3. Temporary tables
4. Use of indexes
4. Locks and Deadlocks – using and troubleshooting
1. Locking in SQL 2000 and isolation levels
2. Simple deadlock scenario
3. Troubleshooting locks and deadlocks
5. Troubleshooting performance problems
1. SQL Profiler to the rescue – how to configure, what to look at
2. SQL Query Analyzer - reading the Execution Plan
3. SQL Server configuration
4. Analyzing index behavior
5. Partitioned tables
6. Some advanced Tips and Tricks
1. TBD

Things I do not plan to discuss at this point:

• Accessing SQL objects programmatically (SQL DMO)
• Using ADO/ADO.NET
• OLAP
• XML Integration
• Reporting and Notification
• Scheduled tasks
• Roles and Permissions

Volunteering in Israel

A few months ago I was asked whether I would be willing to volunteer giving free Math lessons to school students. I accepted willingly and am enjoying it a lot. Since then, my wife has started volunteering giving English lessons as well.
I believe it is important both for society and for the person himself to do some kind of volunteering job once in a while.
In that spirit, if you live in Israel and are looking for a place where your skills might be needed, I suggest you check out this volunteering search engine.

C# Code Snippet Library for Visual Studio 2005

I got this tip from reading Scott Dockendorf’s blog (right here).
It’s been some time that the VB people kept laughing at us C#-ers that they have much more code snippets for VB within VS 2005 than we have for C#. Well, not anymore - check this out.
I guess it’s going to take me some time to get to know all the shortcuts, but I’m sure going to enjoy it a lot

Quaero - where are you???

Almost a year ago (April 2005), French President Jacques Chirac announced his backing of a new Euro-centric search engine called Quaero (which means ‘I am looking for' in Latin). The idea is to build a new search engine which should be capable to compete with MSN, Yahoo! and even Google. Tons of very important European companies are involved in this project, as well as the French and German governments.

Mid January 2006, there was some loud buzz in the air about this project. So loud, in fact, that they decided to get into hiding: the small and unconvincing beta website they had in place has been removed and they came with a statement that around the end of January 2006 they will come out with a new statement (actually a release was expected). Look here for a summary of the latest news.

Today is the 15^th of February and I’m still waiting for such a statement…

As a side note, I’m a little ambivalent about such projects. I don’t like the idea of geologically differentiating core web parts such as search engines. I mean, come on! The past decade has seen such an improvement in accessing data by means of the Internet. Why go back? Also, I don’t like the idea of having governments messing with my searches. We’ve seen what they are capable of (see the China-Google story lately). On the other hand, I’m not sure that having companies being responsible for such an important asset being a much better solution. After all, if my interests as customers will not comply with their interests as profit-thirsty company, we all know what they will opt for (see the China-Google story again…).

IMHO there is an urgent need for international legislation of the search engines issue. I have no idea how this can be done (I don’t know much in international law), but that should be the direction. There should be a set of international laws to which each globally-available search engine should comply. Then, an international committee (with concrete punishment capabilities), would be responsible to make sure all companies/governments managing search engines would abide to these laws.

Build a Data Access Layer in less than 15 minutes

To be honest, I still haven’t decided whether I’m an ORM-guy or a traditional DAL kind of person. Until now I have always worked with regular DAL, and due to various organizational constraints, haven’t had the opportunity to thoroughly check out using ORMs in real-life projects. My heart tends to go towards ORMs due to the OO appeal, but my experience with SQL performance issues forces me to remain skeptical until I’ve seen it work.

Anyways, it seems that if you’re still on the traditional DAL side of the map, CodeSmith has a very interesting solution for building strongly-typed DALs in a very fast and easy manner. I’ll surely try this one out next time I have a new DAL to work on!

An extension to Matlab's Image & Video Toolbox

Piotr Dollar (I swear that’s his name) has written a very nice set of extensions for Matlab’s Image & Video Toolbox.
It includes a whole bunch of existing algorithms with serious performance optimizations, as well as some that are missing in Matlab’s toolbox.

Thanks man, it’s really helpful for my MSc research project!

AllOfMP3

[UPDATE: It's not that simple. Apparently, although they are legal strictly speaking, they don't seem to be very fair. Bottom line is - there is a very big chance that the artist you're downloading songs of will not see a dime from it.
I was a little skeptical, so I started Googling with "AllOfMp3 scam". Take a look at one of the results... Look also at this...
I personally intend to liquidate the remaining 10$ or so I have left in my account and I doubt I will renew it. I really don't like the way the music companies extort us with their prices, but that doesn't mean I want to steal the music. I think that a more reasonable pricing should be available. I know some companies have started with new approaches (such as Napster), but they all sell protected files, which is not practical to me.

So I'm back to square one - looking for a good, reliable, legal, not too expensive (and fair) source of digitally available music...]

I’ve discovered this amazing site yesterday. It’s a legal songs download site, and it’s awesome!!! Their pricing makes so much more sense than all the other sites. They charge you by the byte – the larger the file you download, the more you pay. In addition, for most songs you can choose the exact encoding parameters: encoder, bitrate, other encoder-specific parameters. In the end, you end up paying much much less for your songs. You can roughly count on 10 cent/song with 192kbps encoding. Ah, I almost forgot, the download rate I get is over 160KB/s on average.

Did I mention I liked it, A LOT ?!?!

[Update: One thing I think they should improve is their search capabilities. Currently you can only search by artist, album or song. With such an extensive amount of data, I would expect much more flexible and advanced searching options. Hope this will improve over time...]

AIBO and QRIO - May you rest in peace...

For those who haven’t had the opportunity to meet AIBO or QRIO, I suggest you run looking at the video clips on Sony’s site, because Sony announced recently that they are stopping production of both.

I have had the opportunity to play a bit with AIBO, when Ehud Sharlin came to give a talk at our CS Seminar. This little robot-dog was really impressive. Put some fur on it, and you might really get attached to it! Being a dog-lover, I can’t say it’s the same as the real thing. However, being an AI fan, I must say it’s really impressive and a real loss for the AI community.

XOR operator in C#

There is a XOR operator in C#!!!
For some reason, it’s impossible to find it by searching for “XOR” in MSDN (either in the index or using the search). I don’t know why they’re trying to hide that…
So anyway, it’s a simple bitwise operator: ^
So:

C = A XOR B should be written: C = A^B;

(and there’s of course the complementary ^= which also performs an assignment)

Bringing your PC to a technician...

YNET published today a very interesting and sad article.

They took completely good computers, incorporated one technical problem to each of them and brought them to 6 different PC technical labs.
If you want the full details – have a go at the article on their site.

For those who want the summary (or don’t read Hebrew):

First technical problem: graphics card not right in its slot.
Results: 2 labs fixed the problem with no cost. 1 lab replaced the graphics card with a new one and demanded approx. 30$ for it.

Second technical problem: some of the memory was replaced with corrupted memory, which caused the computer to fail startup.
Results: 2 labs claimed the motherboard AND CPU needed to be replaced. 1 lab simply recommended upgrading to a completely new computer.

Need I say more?

Matlab number formatting

This is an issue that has annoyed many (me included).
Matlab, by default, displays floating point numbers in their ‘short’ format. That is, the number ‘0.181096612595921’ will be displayed in the Command Window as ‘0.1811’. The problem is that you have no indication whether that’s the whole number, or like in this case, only a rounded ‘short’ version of it. Also, it’s not very obvious how this should be changed in the environment (the Matlab Help is far from being the best in town).

So, for those of you who are interested, here it is:

1. Changing the format in using the Preferences (these changes will remain forever, until you change them again):
1. Changing the Command Window display:

Click on the menu: File è Preferences. Choose ‘Command Window’ and change the ‘Numeric Format’ combo.

2. Changing the Array Editor:

Click on the menu: File è Preferences. Choose ‘Array Editor’ and change the ‘Numeric Format’ combo.

2. Changing the format for the current session only: use the ‘format’ function. For example:

‘format long g’ – will display the number above as ‘0.181096612595921’,

Whereas ‘format short g’ will display it as ‘0.1811’.

For details about the various options, check the help on the ‘format’ function.

Wireless router

That’s it, I’ve had it! I’ve been suffering from my Wireless Broadband Router for the past half year and enough is enough!
When I bought my laptop last summer, I also bought a wireless router to go with it, of course. Not knowing what router’s best, I let the guy who sold me the computer to choose the router. So I ended up with a router by Edimax (BR6114WG to be accurate).
At first, everything went smoothly. The installation was fairly easy and I was relatively happy. For the first time in my live I had the possibility to work from my back yard, so I was ready to close an eye on the disconnections I experienced once in a while.
Then, with time, I encountered several problems:

1. The wireless network is really unreliable. Even from short distances, with little disturbance, the connection gets disconnected once in a while.
2. If the router remains on for several days (typically around 3 days), it suddenly stops providing Internet connection.
3. On two occasions the router simply dropped dead. The first one required a complete replacement (it took them 2 weeks to reach to this conclusion). On the second occasion, it suddenly came back to life after a few hours.
4. Once in a while, the router becomes ‘almost dead’. I won’t get into details, but just day that to revive it I had to unplug/plug it to the electricity several times in a row.

In the end, I decided that I’ve been wasting way too much time on this stupid piece of plastic. I checked out the user reviews in Zap and decided to buy a new 3COM router with 3Yrs guarantee. It’s 20-30% more expensive, but if it will save me the administration time, it’s certainly worth it.

I received it today. The installation was slightly more lengthy than with the Edimax (although the documentation is much more professional). I’m happy to say that it’s now up and running. I’ve got a good feeling about this – let’s hope I’m not wrong!

French Onion Soup

Yet another soup I really like for cold winter days. It’s extremely simple, and if you give it enough time it’s wonderful!
Last time I made it I used leek (‘kresha’ in Hebrew) instead of onions and it turned out to be maybe even better.

Ingredients:

• 4-8 Onions depending on their size. Could be replaced by the same quantity of leeks
• 50gr. Yellow butter
• 2 soup spoons of white wheat
• 1 liter of Chicken Bouillon
• 1/2 cup white wine
• 1/4 cup Brandy
• Salt, peper, Chili powder

Directions:

• Cut the onions in stripes. I simply cut them in half and then cut each half in thin slices.
• Fry the onions in the butter, using a high cooking pan. Make sure you keep the fire LOW! It should fry for around an hour, maybe even more, and still not burn. Keep stirring once in a while. You must continue frying until the onions are almost completely melted, transparent, with a slight yellow tint (not burned!). Most of the time I leave it on the fire with closed top (except for when I stir, obviously…).
• Add the wheat and stir thoroughly. Let them fry with the wheat for 1-2 minutes. You must keep stirring otherwise the wheat will burn.
• Add the bouillon and the wine. I usually warm the bouillon up beforehand, but it’s not a must. If you don’t have enough liquid to cover all the onions, you can complement with warm water (or more bouillon). Turn the fire up until it boils.
• Lower the fire and keep it at boiling temperature for another ½ hour (sometimes a little more). Make sure you continue to stir once in a while; otherwise the onions get stuck on the bottom and… burn.
• Add the Chili powder, salt and pepper – according to your taste
• Stir and leave on the fire for a couple of minutes
• Take off the fire and add the brandy, stir
• Mission accomplished

Side note: Chili powder is not something you’ll see often in an onion soup. I tried it once, and found it adds a special taste to the soup, makes it interesting. It also gives it a nice orange-red color (that is not due to over-burning the onions). I’ve never made an onion soup without it since then!

Building a DAL using Strongly Typed TableAdapters and DataTables - by Scott Guthrie

Scott Guthrie (a.k.a. ScottGu) is starting a series of tutorial postings on common data binding design patterns. He’s definitely done a great start with his first post about Building a DAL using Strongly Typed TableAdapters and DataTables.

Thanks Scott!

Mother of all soups - the Chicken Bouillon and a Bouillon-based vegetable soup

Almost all the soups I do are best made with a basis of a chicken bouillon. In some cases, you could do with just water, but usually the bouillon changes the soup from ‘good’ to ‘excellent’. I never use flavoring powders and the like. They are full of mono-sodium glutamate, which is unhealthy and sometimes difficult to digest…

Clear Chicken Bouillon

Ingredients:

• ~2 kg chicken parts. I usually prefer the neck. If I can’t find necks I use wings. Chicken comes out much better than turkey, in my opinion.
• 1 large celery root
• 1 or 2 parsley roots
• Various herbs for a “bouquet garnis”. I use mostly parsley and celery but feel free to use also coriander, dill or whatever you like.
• (optional) 1 or 2 small dried hot peppers
• 4-5 bay-tree leaves (“Laurier” in French or “Dafna” in Hebrew or “Laurus nobilis” in Latin)

Directions:

• Put all the ingredients in a large, high cooking pan.
• Fill with water such that all the ingredients are totally covered.
• Let it boil. Beware – the content has a tendency to overflow when the fire is too high. If it does overflow, it becomes really messy, so once it reaches boiling temperature, make sure to lower the heat.
• Leave it at boiling temperature for 2-3 hours (more towards 3..). Stir once in a while and make sure everything is covered by water. During the whole cooking, leave the cover almost closed (i.e. leave a small gap for the superfluous steam to get out). Some notes should be made at this point:
• If you leave the cover completely open, the soup will dry out to quickly and you’ll be left with too few liquid.
• If you leave the cover completely closed, there’s a good chance it will overflow. Don’t blame me at that point!
• If you stop cooking too early, the soup won’t have enough tastes.
• When it’s ready, transfer it to a large container, preferably one that is transparent and has a thin bottom (explanations why in a second). Make sure to filter out everything: the chicken, the herbs and especially the hot peppers. I usually give the chicken to my dog and dispose of all the rest.
• You may have some remaining residue in the new container – don’t worry about it for the moment.
• Leave it in the refrigerator for several hours (I leave it for the night).
• In the morning, what you should have in your container is a somewhat jelly-like substance. On the bottom you should see the entire residue, and on the top there is a layer of pure fat. Sometimes the fat can become creamy of even crusty; sometimes it’s much like heavy oil.
• The idea is to keep the jelly and drop the rest (fat and residue). Since the bouillon has become jelly, it’s a really easy job:
• With a spoon you scratch all the fat and dispose of it.
• Then you can extract the bouillon while leaving the residue in the container.
• That’s it! Now you have a strong, clear and rather low in fat bouillon. You could either keep it in small containers in the freezer for later use (either in soups or other dishes) or make some really good soup out of it right away!!

Very Simple Vegetable (but not Vegetarian) Soup

Ingredients:

• Strong, fresh, chicken bouillon
• Veggies J . I usually use the following:
• 3-4 pealed, sliced carrots
• 2 leeks (or 2 onions)
• Pumpkin (in cubes)
• 2-3 cloves of garlic, cut into slices
• (optional) half a cauliflower (“Kruvit” in Hebrew), in small pieces

Directions:

• Just put everything in a large cooking pan. Make sure there is enough bouillon to cover all the vegetables. Let it boil and leave it at boiling temperature for about 1 hour. Don’t let it overflow!
• When it’s ready, you can add spices. I would suggest only some salt. Since here in Israel the chicken comes already salted due to the koshering process, I actually don’t add anything.

Twist:

• If you want to give it an oriental taste, you can add the following spices (all or part).:
• Coriander (I’d add it to the bouillon)
• Cumin
• Turmeric
• A tiny bit of Cardamom (“Hel” in Hebrew)
• 2-3 Cloves (“Ziporen” in Hebrew) – I’d add it to the bouillon as well

Well, I think that’s it for now. The next soups (not necessarily in that order) will be: “French Onion Soup”, “Rich Meat and Lentil Soup” and maybe “Simple Pea Soup”.

Bon Appetit J

Soups

One of the few things I really like about winter is the soups. So since the winter is now really showing, here in Israel, I’ve decided to start posting a few messages with the recipes of the soups I like (and prepare) the most. If you happen to try preparing them, I'd be happy to hear how it went (for the good and for the bad).

Nasty SQL Code

Oren Eini (or is his name Ayende Rahien? I'm a little confused about that) has a very nice blog I like to read. He must be going through a really hard time, trying to deal with extremely crappy code lately. I suggest you take a look at his 3 pieces of nasty SQL code (here, here and here). It’s worth it!

I'm back to IE

A few weeks ago I posted that I’ve decided to try using FireFox, with the idea of opening anything that is not sufficiently supported by FireFox with the IETab. Well, I’m sorry to day that this experience didn’t prove successful for me, and I changed my default browser back to IE.
Just to name a few reasons:
1. IETab Pop-up blocking – there is no way (that I could find) to define on which sites you want to allow pop-up blocking and which not when you’re using the IETab. You must set pop-up blocking to either ON or OFF for ALL the sites being viewed with the IETab. This is the most significant reason why I disliked the IETab. As Murphy would have wanted it, most of the sites I need to view with the IETab (mostly but not exclusively bank account sites) use pop-ups. So I do need to allow pop-ups, but I certainly don’t want to allow them for all the sites viewed with the IETab!
2. Performance – odd isn’t it? Well, some sites take up quite a lot of CPU with all their flash publicities and all. I don’t like it, but since I like the content I live with it. One of these sites is YNET. The problem with FireFox, is that even when the browser is minimized, it takes up a significant amount of CPU when these sites are open. With IE, once the browser is minimized, the CPU drops to normal and is free for my other stuff.
3. Defining sites that should be displayed with IETab – the list is a list of exact urls. You can’t define web sites. This has proven to be very annoying.
4. I don’t remember exact examples at the moment, but I did encounter some sites that didn’t look well either with the IETab or without it.

Until these issues are not fixed, I’m sorry to say I’ll have to continue using IE. I know that I could use both, and each time choose the best one, but I hate working that way.

(Right before sending this post I discovered there is a new IETab release. I'll try and play with it some time soon. Maybe it will make me change my mind?)

Network resource KILLERS

I’m not a Web programmer, so I can’t say how you should write web sites. I do, however, have enough experience in software development and networking in order to know some of the thing you should not do.
I had to play quite a lot with the Windows Event Viewer lately. While working, I noticed an ever recurring warning that goes like so:
“TCP/IP has reached the security limit imposed on the number of concurrent TCP connect attempts.”
Since it even happened very shortly after having rebooted my system, I decided to track down the source of it. Here are results of playing with the netstat utility:

1. No particular application open. Netstat returns 3 TCP connection owned by the following processes: MsnMsngr.exe, Skype.exe and gnotify.exe. Makes sense, doesn’t it

2. I opened one instance of Internet Explorer and entered YNET. Now, in addition to the above-mentioned 3 connections I had 61 other connections, all owned by IEXPLORE.EXE

That’s definitely a huge WTF ?!?!?! You open one single web site and you’re swamped with so many connections that in the end the OS has to start imposing limitations because it believes you may be running a virus/worm or some other threatening application.
Needless to say that when I checked with FireFox the behavior was more or less the same (didn’t count exactly the number of connections but it’s close enough).
Also, most other sites I checked, although they do use quite a lot of connections, it’s still not that much.

.NET 2.0 Goodies

I’ve finally started to work with .NET 2.0. I know, it’s considered very late, but I had my reasons...
Anyway, here is one line of code I had to use today that makes use of several new features I find very cool (and useful):

Array.ForEach(threads, delegate(Thread t) { t.Join(); });

Assuming you have an array of threads named ‘threads’, this line will make sure you wait for all of them to complete. It uses Generics, Anonymous Methods and Anonymous Delegates all at once…

IDesign's Serializer

In one of my first posts I reported on a MessageLoop object I wrote and posted to the CodeProject.
Today I discovered that at IDesign they wrote a Synchronizer object (go to the link and search for the word 'Synchronizer') that provides the same functionality. Their implementation has the advantage to implement the ISynchronizeInvoke interface, which means the solution is much more complete. But then again it means that my implementation, is much simpler. Also, I have added some features to my implementation that you won't find in Synchronizer:
1. Priority on messages (i.e. make it possible to execute a message before older messages)
2. Limit the queue size - when the limit is reached, new messages are being dropped.
I haven't added these features to the published version yet, but for my personal needs they are important, so I'll keep using it (although I might merge both some time in the future).

Thanks to my friend Moshe Anconina for pointing this out to me.

I burned my wife...

Believe it or not, but last week I actually burned my wife. After having served her a boiling cup of tea, I made a wrong movement and the cup turned over, spilling all of the boiling liquid on her leg.
It was a nightmare! Even though it took us only 2 or 3 seconds to get her pants off and put enormous quantities of water on the burn, the damage was already done. She has a 2nd level burn on an area of 20x40 centimeters. She said that the pain during the first day was worse than giving birth. And she gave birth without epidural!!!
I can't emphasize how sorry I am and how bad I feel about this. She's not angry, of course, for it was obviously and accident, but I still feel I would have better liked doing it to myself on both legs than having her suffering this.

Maybe the most infuriating thing it that if the water was just a few degrees lower, the burn would have been much lighter. So what I do from now on, and that's my tip as an experienced leg-burner: whenever I prepare some hot beverage, I make sure to add a bit of cold water before serving it!!!

I sincerely hope that maybe this post will help avoid such accidents for other people.

Had some problem with my Atom feed

There was a problem in my last post - it caused my Atom XML to be corrupted, so anyone who reads my posts through RSS got dissapointed.
I fixed it and apologize in the name of Blogger.com (it was a post automatically generated by sending an email to Blogger, to which I made small modifications).

Visual SourceSafe madness

Say you want to install Visual Studio 2005 WITH the new Visual Source Safe client.
Well, apparently it’s much more complicated than you’d have thought.
Just go here and you’ll see what I mean.

Amazing!!!

(Thanks to my friend Gil for pointing this out to me)

FireFox dillema

In the past I have tried using FireFox. It left me with mixed feelings. On one hand, I liked the tabs A LOT and felt the configuration options are much more usable than with IE. I also felt that the way favorites could be saved in groups (also by saving all open tabs in one click) was particularly helpful to me. But on the other hand I was disappointed to see that: a) the memory footprint was very similar to that of IE (sometimes higher sometimes lower) and b) Since many sites are still incompatible with FireFox, I had to keep using both all the time.

In the end I decided that there was no point wasting my time juggling between them all the time, so I crawled back (with my tail between my legs) to our not-so-good but certainly old IE and stopped using FireFox.

Until today.

Today I found out there is an add-on that seems to be able to solve that exact problem: the IETab. It provides you with the possibility to view web pages in an IE tab from within FF. You can switch between using the IE engine or the FF engine with a single click, or even configure a list of websites that will be opened in an IE tab by default. This seems to be exactly what I needed!

Bottom line: I’m going to download the latest version of FireFox right-away, and install it with the IETab. Hopefully it will make it possible for me to enjoy both worlds and live happily ever after…