Getting information in Matlab

One of the things I really hate with Matlab is that when you don’t know exactly what you’re looking for, you stand a good chance not to find it. Hmm, this doesn’t sound right, let me rephrase: when you don’t know how what you’re looking for is called in Matlab, you stand a good chance not to find it.
Today, for example, I wanted to retrieve the file that implements a specific function. Since I’m using several free toolboxes I downloaded from the internet, I wanted to know whether the function is part of the Matlab installation or of one of the toolboxes. I started by typing:

>> help pca

Which gave me a complete description for the 'pca' function. Yet, it didn’t provide its source. I couldn’t find any allusion to this function in the Help, which indicates there is a good chance that this method comes from an external toolbox. Note however that it’s not always the case – some functions that are part of the installation are not easy to locate in the Help window, even if you know their exact name.
After several minutes of frustration I discovered that the following line:

>> which pca

returns the full path of the file that implements the function ‘pca’ (and indeed it was from an external toolbox).

My frustration was – how the h… am I supposed to know that the function that provides this information is called ‘which’ ??? My situation was even better than most cases, because I vaguely remembered that there is such a function. Sometimes you don’t even know exactly how you can do what you actually want to do, and then looking it out in the Help becomes a real nightmare.

That’s all, just had to get this off my chest…

SQL for Developers Tips and Tricks: 1. Introduction

Lately I’ve been mostly working on research projects related to Machine Learning (both for my thesis and as a private contractor). Therefore, my ability to blog on development issues I’m encountering is somewhat limited. Therefore I have decided to start blogging about stuff I’ve learned and used along the years, both to keep it as a written memory and maybe to help some readers…

So for starter, I’ve decided to summarize various SQL issues I liked/learned/used. Some I got from fellow developers or DBA’s and some I had to learn on my own (sometimes at great personal risk to become completely crazy). I’ll try to do it in an organized fashion, each blog focusing on some other subject. The vantage point is that of a software developer not a DBA. I have seen many cases where a developer coming from C/C++ started using SQL. The first few days she invests in getting a good grip at the basic syntax (SELECT, INSERT, UPDATE, …). Then she starts writing stored procedures and learns some SQL programming concepts (like CURSORs). At some point, she starts getting acquainted with Views and Functions and learns various helpful built-in SQL functions. Then, slowly but surely, the problems start to arise, usually performance. Once the performance issues are more or less under control, there comes a time to design a new database. This new database inevitably includes many flaws, which will have to be dealt with at some point or another.

I have gone through this, and many good programmers I know have gone through a similar sequence. The purpose of this set of posts is to try and give the programmer a set of tools and knowledge base that can help her go through this ordeal with more ease and confidence. This is by no means a complete tutorial, but I still hope it will be of help. If there are any specific requests, please send a comment with the details. If I know the answer, I’ll try to asses it in the future posts.

One last thing before I present the list of issues I intend to cover – since I’m still too short in hands-on experience with SQL2005, most of the information will be provided with SQL2000 in mind. I guess that most of the issues are applicable to SQL2005 as well, although the new SQL version certainly brings a whole series of new issues (with the CLR and all) as well as additional and revised tools. Also, these posts will assume familiarity with the basic SQL syntax (SELECT, INSERT, UPDATE).

Following is the list of issues I plan to asses. This is a preliminary list and may be altered along the way, but should give you some idea of what to expect.

1. Introduction
2. Basic tools you should keep at the ready
1. Helpful web sites
2. SQL Query Analyzer – things too many people learn too late
3. SQL Profiler
3. SQL Programming for Performance – basic programming tips that can help reduce future performance problems
1. Similar code that runs faster
2. Cursors
3. Temporary tables
4. Use of indexes
4. Locks and Deadlocks – using and troubleshooting
1. Locking in SQL 2000 and isolation levels
2. Simple deadlock scenario
3. Troubleshooting locks and deadlocks
5. Troubleshooting performance problems
1. SQL Profiler to the rescue – how to configure, what to look at
2. SQL Query Analyzer - reading the Execution Plan
3. SQL Server configuration
4. Analyzing index behavior
5. Partitioned tables
6. Some advanced Tips and Tricks
1. TBD

Things I do not plan to discuss at this point:

• Accessing SQL objects programmatically (SQL DMO)
• Using ADO/ADO.NET
• OLAP
• XML Integration
• Reporting and Notification
• Scheduled tasks
• Roles and Permissions

Volunteering in Israel

A few months ago I was asked whether I would be willing to volunteer giving free Math lessons to school students. I accepted willingly and am enjoying it a lot. Since then, my wife has started volunteering giving English lessons as well.
I believe it is important both for society and for the person himself to do some kind of volunteering job once in a while.
In that spirit, if you live in Israel and are looking for a place where your skills might be needed, I suggest you check out this volunteering search engine.

C# Code Snippet Library for Visual Studio 2005

I got this tip from reading Scott Dockendorf’s blog (right here).
It’s been some time that the VB people kept laughing at us C#-ers that they have much more code snippets for VB within VS 2005 than we have for C#. Well, not anymore - check this out.
I guess it’s going to take me some time to get to know all the shortcuts, but I’m sure going to enjoy it a lot

Quaero - where are you???

Almost a year ago (April 2005), French President Jacques Chirac announced his backing of a new Euro-centric search engine called Quaero (which means ‘I am looking for' in Latin). The idea is to build a new search engine which should be capable to compete with MSN, Yahoo! and even Google. Tons of very important European companies are involved in this project, as well as the French and German governments.

Mid January 2006, there was some loud buzz in the air about this project. So loud, in fact, that they decided to get into hiding: the small and unconvincing beta website they had in place has been removed and they came with a statement that around the end of January 2006 they will come out with a new statement (actually a release was expected). Look here for a summary of the latest news.

Today is the 15^th of February and I’m still waiting for such a statement…

As a side note, I’m a little ambivalent about such projects. I don’t like the idea of geologically differentiating core web parts such as search engines. I mean, come on! The past decade has seen such an improvement in accessing data by means of the Internet. Why go back? Also, I don’t like the idea of having governments messing with my searches. We’ve seen what they are capable of (see the China-Google story lately). On the other hand, I’m not sure that having companies being responsible for such an important asset being a much better solution. After all, if my interests as customers will not comply with their interests as profit-thirsty company, we all know what they will opt for (see the China-Google story again…).

IMHO there is an urgent need for international legislation of the search engines issue. I have no idea how this can be done (I don’t know much in international law), but that should be the direction. There should be a set of international laws to which each globally-available search engine should comply. Then, an international committee (with concrete punishment capabilities), would be responsible to make sure all companies/governments managing search engines would abide to these laws.

Build a Data Access Layer in less than 15 minutes

To be honest, I still haven’t decided whether I’m an ORM-guy or a traditional DAL kind of person. Until now I have always worked with regular DAL, and due to various organizational constraints, haven’t had the opportunity to thoroughly check out using ORMs in real-life projects. My heart tends to go towards ORMs due to the OO appeal, but my experience with SQL performance issues forces me to remain skeptical until I’ve seen it work.

Anyways, it seems that if you’re still on the traditional DAL side of the map, CodeSmith has a very interesting solution for building strongly-typed DALs in a very fast and easy manner. I’ll surely try this one out next time I have a new DAL to work on!

An extension to Matlab's Image & Video Toolbox

Piotr Dollar (I swear that’s his name) has written a very nice set of extensions for Matlab’s Image & Video Toolbox.
It includes a whole bunch of existing algorithms with serious performance optimizations, as well as some that are missing in Matlab’s toolbox.

Thanks man, it’s really helpful for my MSc research project!

AllOfMP3

[UPDATE: It's not that simple. Apparently, although they are legal strictly speaking, they don't seem to be very fair. Bottom line is - there is a very big chance that the artist you're downloading songs of will not see a dime from it.
I was a little skeptical, so I started Googling with "AllOfMp3 scam". Take a look at one of the results... Look also at this...
I personally intend to liquidate the remaining 10$ or so I have left in my account and I doubt I will renew it. I really don't like the way the music companies extort us with their prices, but that doesn't mean I want to steal the music. I think that a more reasonable pricing should be available. I know some companies have started with new approaches (such as Napster), but they all sell protected files, which is not practical to me.

So I'm back to square one - looking for a good, reliable, legal, not too expensive (and fair) source of digitally available music...]

I’ve discovered this amazing site yesterday. It’s a legal songs download site, and it’s awesome!!! Their pricing makes so much more sense than all the other sites. They charge you by the byte – the larger the file you download, the more you pay. In addition, for most songs you can choose the exact encoding parameters: encoder, bitrate, other encoder-specific parameters. In the end, you end up paying much much less for your songs. You can roughly count on 10 cent/song with 192kbps encoding. Ah, I almost forgot, the download rate I get is over 160KB/s on average.

Did I mention I liked it, A LOT ?!?!

[Update: One thing I think they should improve is their search capabilities. Currently you can only search by artist, album or song. With such an extensive amount of data, I would expect much more flexible and advanced searching options. Hope this will improve over time...]