Wednesday, March 28, 2007

I don't know what this guy is on - but I want some...

This is really hallucinating. If this thing is for real, I want to know what this guy takes - and want some for myself...

Oren Zarif is one of these people who claim to have exceptional, supernatural and X-Files-like powers. Apparently, he likes to get it really hard. According to YNET, he has contacted Mike Tyson, proposing a fight. Zarif , weighting 68 kgs, as far from being a boxer talent as one can be, claims to be able to box with his eyes closed. He proposes a one-on-one, with his eyes completely covered. If Zarif loses, he would pay Tyson 5M$. If he wins, Tyson will have to donate the same amount to the US government for the victims of global terror.

For my English readers, the letter sent to Tyson, in passable English, can be found here. Source - YNET.

WTF ?!?!?

A Million Dollar Contest - For Super-Geeks

Netflix is a very large (largest?) DVD rental company. One of their most important assets is Cinematch, an in-house developed movie recommendation system. Cinematch's purpose is to predict, based on a user's previous rating of movies, the rating a user would give to other movies. The result, of course, is a recommendation of movies that are most likely to suite the user's taste.

Apparently, Cinematch works pretty well, but the guys at Netflix would like it to work even better. So they came up with the Netflix Prize: encourage developers and researchers to come up with an algorithm that improves the quality of rating prediction of Cinematch significantly enough (at least by 10%), by promising a prize of 1,000,000$. To make it even more interesting, and since the whole contest spans over at least 5 years, there is a yearly prize of 50,000$ which will be given to the best solution each year.

Being a Machine Learning freak, I find this contest GREAT! I also like the rules of the contest a lot. Basically, the winning algorithm will have to be made publicly available at the end of the contest.

This kind of initiatives are so great because they encourage the development of new concepts and algorithms, which even if they won't win the first prize, might very well be helpful for various other purposes. I think the best comparison is Fermat's Last Theorem. In his will, Paul Wolfskehl initiated a prize of then 100,000 marks to whomever would be able to prove or disprove Fermat's Last Theorem. This generated a huge interest in the subject, which resulted in an incredibly rich amount of new ideas and whole new areas of mathematics being discovered and researched to this day. I doubt that the Netflix Prize will have the same effect, but I do believe it will give Machine Learning a well-deserved boost.

As far as Netflix is concerned - they can only win from it. The news about this contest should inevitably increase their exposure. If the contest succeeds, and someone manages to provide significantly better results - it will be worth much more than 1M$ for them. If nobody manages to win the contest, then they can heartily claim to be using the best movies matching algorithm human brain could come up with do date. Either way, it's a win-win situation for them.

It's a very difficult task, but I think I'm going to give it a try, as far as time permits...

If you're interested, following are the Terms and Conditions in a Nutshell:

Contest begins October 2, 2006 and continues through at least October 2, 2011.

Contest is open to anyone, anywhere (except certain countries listed below).

You have to register to enter.

Once you register and agree to these Rules, you’ll have access to the Contest training data and qualifying test sets.

To qualify for the $1,000,000 Grand Prize, the accuracy of your submitted predictions on the qualifying set must be at least 10% better than the accuracy Cinematch can achieve on the same training data set at the start of the Contest.

To qualify for a year’s $50,000 Progress Prize the accuracy of any of your submitted predictions that year must be less than or equal to the accuracy value established by the judges the preceding year.

To win and take home either prize, your qualifying submissions must have the largest accuracy improvement verified by the Contest judges, you must share your method with (and non-exclusively license it to) Netflix, and you must describe to the world how you did it and why it works.

For more elaborated information, check out the Netflix Prize page.

Yahoo! Mail to become unlimited in size?

According to YNET, Yahoo! announced that as of May 2007 they will gradually update all Yahoo! Mail accounts to unlimited mailbox sizes. I couldn't find any additional source mentioning this, but I trust YNET didn't make this up.

This isn't surprising, of course. Ever since Google introduced GMail it was obvious that at some point they would announce that the mailbox size has become unlimited. Maybe the surprising thing is that the first company to come with such an announcement is Yahoo!, who's been lagging behind Google with most of the previous advances.

This is, of course, a very welcome announcement, hopefullly to be followed by all the other main webmail suppliers.

Cool!

P.S: I think that with this new development, it's time that the ISP's decide whether they want to continue supplying email services to their customers. If they do - then they should dramatically improve the level of their service (speed and mailbox size). Otherwise, they're really making fools of themselves.

Sunday, March 25, 2007

Joke By Code

I haven't blogged much lately, I know, sorry. Between work, kids, buying a house (!!!!!) and a million other things, I just couldn't find a minute.

Anyway, today I was searching for some good books about SQL Server 2005 programming. I have a lot of experience with SQL 2000, but it's about time I get to know the little baby (who's way past the 'baby' stage and would better be called a teenager) a little more.

So I came accross Murach's SQL Server 2005 for Developers and like everyone, I started skimming through the 6 customer reviews. All reviews were really good, except the last one, which seems to have been written by particularly dissapointed customer. So I decided to have a look at the 3 comments to his review. Here is what I found (quote):

SELECT OneHalfBrain
FROM Name
WHERE LName = `Husain'

results: NULL

It made me laugh (the reviewer is called Munawer Husain, so I assume it was meant personally against him, and not in some stupid racist direction).

Tuesday, March 13, 2007

Multithreaded Computation Support in Matlab - FINALLY!!!

A couple of weeks ago, Mathworks released a new version of Matlab (R2007a). They have, at last, added support for multithreaded computations.
A LOT of entries to my blog come from searches like "matlab dual core", which surprisingly puts a post of mine very high in the results list. The thing is that especially now with the increased usage of dual and quad-core machines, running heavy Matlab computations with a single thread is just a waste of resources. Apparently, the guys at Mathworks were listening to the users, and added support for multi-threaded calculations. It seems to be managed under the hood somehow, and requires changes to the preferences. From the release notes:

"If you run MATLAB on a multiple-CPU system (multiprocessor or multicore), use a new preference to enable multithreaded computation. This can increase MATLAB performance for element-wise and BLAS library computations.
By default the preference is not set, so you must set it to enable multithreaded computation. With the preference enabled, MATLAB automatically specifies the recommended number of computational threads, although you can change that value. On AMD-based Linux platforms, MATLAB supports multithreaded computation, but requires an extra step to change the default BLAS."

Doesn't sound a very nice way to do it, and certainly lacks user control, but it's a start. I didn't upgrade to the new version yet, so I can't talk from experience. Also, the new release seems to include improvements to the Distributed Computing Toolbox, which sound also very interesting.

Sunday, March 11, 2007

Google Image Search API updated (again...)

What can I say - the html format was changed again.
You can download the complete code from my article on CodeProject.
If you just want the update - I just had to change the Regex file, so you can simply replace its content with the following:


imagesRegex: (dyn\x2EImg\x28\x22[^\x22]*\x22,\x22[^\x22]*\x22,\x22(?<code>[^\x22]*)\x22,\x22(?<imgurl>[^\x22]*)\x22,\x22(?<width>[^\x22]*)\x22,\x22(?<height>[^\x22]*)\x22,[^\x29]*)
dataRegex: (?<width>[0-9,]*)\s+x\s+(?<height>[0-9,]*)\s+(pixels\s+){0,1}-\s+(?<size>[0-9,]*)(k)
totalResultsRegex: (?<upperLimit>upperLimit>(\s)*)(?<lastResult>[0-9,]*)([^=])*=(?<maxLimit>maxLimit>(\s)*)(?<totalResultsAvailable>[0-9,]*)

Thursday, March 01, 2007

Is it the end of Internet Freedom? A possible workaround...

Politicians are trying to add more and more limits to the freedom and anonymity we enjoy over the Internet. A few months ago, they wanted to force talkbackers to identify themselves - death sentence to talkbacks. Now, they are seriously talking about forcing users to identify themselves when using adults websites.
It's a pitty really. Instead of searching for ways to improve the quality of the service they provide us, they try to castrate the Internet into something more manageable (for them). Also, the philosophy behind this law they are trying to pass, is so anti-democratic and anti-privacy that it makes me want to cry.

Anyway, there might be a way around this. Check out Tor . It's a technology that basically routes web requests (or any other TCP-based communication) through various nodes in an encrypted form, making it almost impossible to track down the original user.

Wednesday, February 28, 2007

A Personal Letter to Everyone I Know, Knew, Will Ever Know, Think I Know, Think S/He Knows Me - To EVERYONE!!!

 

Dear family, friends, ex-friends, future-friends, acquaintaces, whatever,

That's it, I've had it!

STOP SENDING ME CHAIN LETTERS !!!

When you think AOL/Microsoft/IBM/whatever is monitoring emails, and you forward them in hope for earning a few bucks - I understand you have as much common sense as my 3-year old (or less).

When you think you can save the life of some poor person with cancer who needs AB blood - it's obvious you haven't been around the Internet for the past 10 years.

When you think luck will come to you by spamming others' mailbox - I truely hope you are never going to meet any Hari-Krishna, because you're soo easy to fool.

When you think you make me feel loved and happy by forwarding some 10-pages long 15-year old load of BS - you're oh! so wrong!

So please please please, I don't want to offend you. If you have something to write to me - be my guest, I'd be glad to read it. If it's something you got from someone who got it from someone who... don't think - leave me out of it! I'd better loose one good joke than have yet another 5 spam (sorry - chain) mails in my inbox!!!

Yours sincerely,

Ilan

P.S: If you really can't help it, and still want to forward this junk (not to me!), I suggest you check out the following first:

http://breakthechain.org/
http://www.snopes.com/
http://info.org.il/irrelevant/  (Israeli)

Wrapper for svmpredict.exe

% predictions = libsvmpredict(test_set, model, varargin)
% options:
% -b probability_estimates: whether to predict probability estimates, 0 or
% 1 (default 0); for one-class SVM only 0 is supported
%
% NOTE1: This function actually executes LibSVM's svmpredict tool. For more
% info check out: http://www.csie.ntu.edu.tw/~cjlin/libsvm/
% NOTE2: Although LibSVM supports more types of labels, this wrapper is
% limited to integer labels only.
% NOTE3: This function assumes that svmpredict.exe is located in a directory
% that lies in your system's PATH.
function predictions = libsvmpredict(test_set, model, varargin)
if isempty(varargin)
options = struct([]);
else
options = varargin{1};
end;

% Dump the test set to a temporary file
testfile = tempname;
fid = fopen(testfile, 'wt');
dumpData = zeros(size(test_set,1), 2*size(test_set,2) + 1);
format = '%d %d:%f';
for i=1:size(test_set, 2)
dumpData(:, 2*i) = repmat(i, size(test_set,1), 1);
dumpData(:, 2*i+1) = test_set(:, i);
if i < size(test_set,2)
format = sprintf('%s %%d:%%f', format);
end;
end;
format = sprintf('%s\n', format);
fprintf(fid, format, dumpData');
fclose(fid);

% Dump the model to a temporary file
modelfile = tempname;
fid = fopen(modelfile, 'wt');
fprintf(fid, model');
fclose(fid);

% Run svmpredict with the given model and write the predictions to a temporary
% file
predictionsfile = tempname;
command = 'svmpredict';

if isfield(options, 'b')
command = sprintf('%s -b %d', options.b);
end;

dos(sprintf('%s %s %s %s', command, testfile, modelfile, predictionsfile), '-echo');

% Load the predictions temporary file and return its content
predictions = dlmread(predictionsfile);

return;

Wrapper for svmtrain.exe

% model = libsvmtrain(training_set, options)
%
% options:
% -s svm_type : set type of SVM (default 0)
% 0 -- C-SVC
% 1 -- nu-SVC
% 2 -- one-class SVM
% 3 -- epsilon-SVR
% 4 -- nu-SVR
% -t kernel_type : set type of kernel function (default 2)
% 0 -- linear: u'*v
% 1 -- polynomial: (gamma*u'*v + coef0)^degree
% 2 -- radial basis function: exp(-gamma*|u-v|^2)
% 3 -- sigmoid: tanh(gamma*u'*v + coef0)
% 4 -- precomputed kernel (kernel values in training_set_file)
% -d degree : set degree in kernel function (default 3)
% -g gamma : set gamma in kernel function (default 1/k)
% -r coef0 : set coef0 in kernel function (default 0)
% -c cost : set the parameter C of C-SVC, epsilon-SVR, and nu-SVR (default 1)
% -n nu : set the parameter nu of nu-SVC, one-class SVM, and nu-SVR (default 0.5)
% -p epsilon : set the epsilon in loss function of epsilon-SVR (default 0.1)
% -m cachesize : set cache memory size in MB (default 100)
% -e epsilon : set tolerance of termination criterion (default 0.001)
% -h shrinking: whether to use the shrinking heuristics, 0 or 1 (default 1)
% -b probability_estimates: whether to train a SVC or SVR model for probability estimates, 0 or 1 (default 0)
% -wi weight: set the parameter C of class i to weight*C, for C-SVC (default 1)
% -v n: n-fold cross validation mode
%
% NOTE1: This function actually executes LibSVM's svmtrain tool. For more
% info check out: http://www.csie.ntu.edu.tw/~cjlin/libsvm/
% NOTE2: Although LibSVM supports more types of labels, this wrapper is
% limited to integer labels only.
% NOTE3: This function assumes that svmtrain.exe is located in a directory
% that lies in your system's PATH.
function model = libsvmtrain(training_set, labels, varargin)
if size(training_set,1) ~= size(labels,1)
error('The training_set and labels should have the same number of rows.');
end;
if isempty(varargin)
options = struct([]);
else
options = varargin{1};
end;

% Dump the training set to a temporary file
datafile = tempname;
fid = fopen(datafile, 'wt');
dumpData = zeros(size(training_set,1), 2*size(training_set,2) + 1);
dumpData(:, 1) = labels(:, 1);
format = '%d';
for i=1:size(training_set, 2)
dumpData(:, 2*i) = repmat(i, size(training_set,1), 1);
dumpData(:, 2*i + 1) = training_set(:, i);
format = sprintf('%s %%d:%%f', format);
end;
format = sprintf('%s\n', format);
fprintf(fid, format, dumpData');
fclose(fid);

% Run svmtrain with all given options and write the model to a temporary
% file
modelfile = tempname;
command = 'svmtrain';

if isfield(options, 's')
command = sprintf('%s -s %d', options.s);
end;

if isfield(options, 't')
command = sprintf('%s -t %d', options.t);
end;

if isfield(options, 'd')
command = sprintf('%s -d %f', options.d);
end;

if isfield(options, 'g')
command = sprintf('%s -g %f', options.g);
end;

if isfield(options, 'r')
command = sprintf('%s -r %f', options.r);
end;

if isfield(options, 'c')
command = sprintf('%s -c %f', options.c);
end;

if isfield(options, 'n')
command = sprintf('%s -n %f', options.s);
end;

if isfield(options, 'p')
command = sprintf('%s -p %f', options.p);
end;

if isfield(options, 'm')
command = sprintf('%s -m %f', options.m);
end;

if isfield(options, 'e')
command = sprintf('%s -e %f', options.e);
end;

if isfield(options, 'h')
command = sprintf('%s -h %d', options.h);
end;

if isfield(options, 'b')
command = sprintf('%s -b %d', options.b);
end;

if isfield(options, 'wi')
command = sprintf('%s -wi %f', options.wi);
end;

if isfield(options, 'v')
command = sprintf('%s -v %d', options.v);
end;

dos(sprintf('%s %s %s', command, datafile, modelfile), '-echo');

% Load the model temporary file and return its content
fid = fopen(modelfile, 'r');
model = fread(fid, '*char')';
fclose(fid);

% For some reason, the data read has superfluous newlines (it seems to have
% doubled the newline characters or something like that).
model = regexprep(model, '\r', '');

return;

LibSVM Matlab Wrappers

As I mentioned in previous posts, I've been using SVMLight for quite some time now. It's been good while it lasted, but I'm now in the search for a stronger SVM library, especially one that handles multiple labels well. I've been introduced to LibSVM, which seems very promising. In theory, they have libraries for various systems and languages, including Matlab. However, I've had quite some trouble making it work with Matlab, so instead of breaking my head into solving the problem, I've decided to work around it in a quick-and-dirty manner (my favorite...).

What I did, is simply wrap the LibSVM's executables in a simple way, so it can be executed using Matlab functions (given the executables are located somewhere in your PATH).

The following two posts will include a wrapper for svmtrain.exe and svmpredict.exe

Please let me know if you encounter any problems using them.

Ah, of course, they are provided AS IS, no warranty or anything alike. I also don't take any credit for it, nor any responsibility as to how/who/when/where/etc. it can be used. For this - please check with the authors of LibSVM.

Monday, February 26, 2007

NTFS File encryption does not like virtual drives

I like the NTFS file encryption feature - you simply define a folder as being encrypted, and any susequent file you add to it is automatically encrypted. This means that, depending on permissions, other users may see that the file exists, but they won't be able to get the content. Even the system Administrator won't be able to read the file's content!
(There are various other goodies related to encrypted files, such that you can't copy them and such, but that's besides the point).

But I also like virtual drives. For example - my 3 PC's have a virtual drive called Z: that maps to a shared folder in one of the machines (called C:\Shared - to be precise).

So I had a couple of files in an encrypted folder (accidentally on the machine that hosts the folder mapped to by Z), and wanted to transfer them to another machine. I sent them to Z, and then I wanted to decypt them so I would be able to retrieve them from my second machine.
Just to make sure the scenario is clear: I'm connected with the correct user (the one who encrypted the files), and simply want to decrypt the files. The only thing is that I want to do it from a virtual drive (Z:).

 

BABOOM!!!

"An error occurred applying attributes to the file:
....
The system cannot find the path specified."

Of course, once I tried to decrypt from C:\Shared instead of Z:\ - everything went fine.

Not nice, really not nice.

Sunday, February 25, 2007

My daughter can do magic !!!

Purim is getting near, and this year, our eldest daughter Gal (almost 3) is going to dress up as a fairy. So yesterday we decided to try out the costume - pink dress, purple skirt on top of it, purple wings, silver crown and, inevitably - silver star-tipped wand.
Once she was completely dressed she started playing with her wand, in it went more or less like this:

Gal: Hocus, pocus, bili-bili-bocus: Grandma and Grandpa.
(nothing happens)
Gal: Where are Grandma and Grandpa ?
Ma&Pa: Huh?
Gal: Where are Grandma and Grandpa ???
(our little brains try to understand why she suddenly starts asking about her grandparents. after a minute or so, the smart of us gets the point)
Ma: Were you expecting Grandma and Grandpa ?
Gal: Yes - I said "Hocus, pocus, bili-bili-bocus" and Grandma and Grandpa didn't come!!!

(At this point I understand what's going on. So I step aside, secretly call Grandma and Grandpa, and ask them to come, and give us a call when they're outside so she can conjure them again, and this time succeed).

Ma: You know, you need to practice a lot for this kind of spells to succeed. Come on, let's practice...

(After half an hour, and a phone call from Grandma and Grandpa telling us they are outside)

Ma: Come on, let's try conjuring Grandma and Grandpa again
Gal: Hocus, pocus, bili-bili-bocus: Grandma and Grandpa.

The face she did when she saw them entering the house, was worth all the gold in the world...

Tuesday, February 20, 2007

Reflector 5.0 is out - awesome!!!

Usually, you don't feel much difference when Lutz Roeder releases a new version of his can't-live-without-it tool. You open it, it says that there is a new version, you click OK, and you don't feel any difference. You're using a new version of Reflector, but don't see where the big deal was.

Today, however, things went completely differently. I downloaded the new version (using the Automatic Update feature which works like a charm), and was already surprised that the downloaded files was so big (1 MB).
Then I opened it... This guy is a genious:

  1. Integrated help from MSDN - when you look at a method's decompilation, you automatically see, at the bottom of the screen, the MSDN information about that method. Even cooler - the links inside that pseudo-MDSN pane work inside Reflector, and navigate to the correct place showing you the decompilation automatically!
  2. Expand Methods - When you look at a class, there is a link at the bottom called "Expand Methods" which will expand all the methods in one screen, effectively providing you with a complete decompiled class.
  3. Enhanced Analyzer feature - I LOVE the "Assigned By" feature, which shows you which methods change a field!
  4. Integrated search in MSDN and Google (I wonder how long it will take for Microsoft and Yahoo! to go complaining about the lack of configurability of the search engine...)
  5. Shell integration - run reflector.exe /register in a command prompt. Do it! NOW!!! You did it? Sure?!?! Well, assuming you're smart enough to have taken this advice - from now on, when you right-click on a managed dll, you'll have a "Browse with .NET Reflector" option, which will automagically open the dll with, huh, well Reflector of course. Actually, you don't even have to go through right-click - it's set as default when you double-click the dll...
  6. More features here. I wouldn't be able to put it better - The Best Tool Ever Got Even Better!!!

Cool!

Thanks Yossi for pointing it out.

Sunday, February 18, 2007

Microsoft Office errors are soooo descriptive....

Source: http://www.microsoft.com/products/ee/transform.aspx?ProdName=Office&EvtSrc=OfficeErrorMessages&EvtID=10046&LCID=1033&ProdVer=11.0

 

Details

Product:
Office

Event ID:
10046

Source:
OfficeErrorMessages

Version:
11.0

Message:
Links to additional information about the error message you saw are available below.


Currently there are no Office Assistance articles available for this error message.

Wednesday, February 14, 2007

What's up with YNet ???

For the past few days, every once in a while, when I open YNet, my browser takes up 100% CPU and I have to kill my whole iexplorer session to continue working properly. I've installed IE7 only a few weeks ago, so I don't know whether it's IE7 or YNet's fault, but it's starting to really get on my nerves!!!

Tuesday, February 13, 2007

4th Israeli bloggers dinner

If you're interested, make sure to let Omer know about it.
I really hope I'll be able to make it...

Anyways - kuddos to Omer for setting this up, again. It also looks like the deal he managed to get is a good one - I'm hungry only from thinking about it...

Political confession

I'm generally a left-wing voter - the two last elections I voted Meretz. I can't say I'm happy with their behavior, nor that I agree with all their opinions, but it's as close as it gets.

Anyway, I must admit that in the 18+ years I've been living in Israel, there has been no minister as good at his job as Ariel Atias, our minister for communications - from Shas - quite the oposite of my political beliefs. This guy is AMAZING - he is 100% focussed on improving service to the citizen, and doesn't mind fighting all the big communications, in most cases ending with the upper hand.
Just to name a few:

  • Force companies to make it very clear to the customer, while signing a contract, what happens at the end of the contract (not just in small letters).
  • Limit the ability of companies to force customers to buy their phones through them.
  • Forced Bezeq to provide a free phone book over the internet (144).
  • Asked the companies to give reductions to people living in the north, during the war.
  • Still working on making our phone numbers company-free (i.e. changing company doesn't force you to change your number). This has been postponed several times, but I hope it will succeed in the end.
  • Opened the stationary phone market to concurrence.
  • Force companies to provide 3 non-paid seconds before being routed to a voice message box.
  • His new baby - force companies to announce customers BEFORE they increase their prices.

And that's just part of it. A good article about him was recently published in YNet. Give us a handful ministers and politicians like him - and this could really be a nice place to live.

I was born on the same day as Albert Einstein

Yep, that's right, just 96 years later. You can see it all here, Together with Billy Crystal, Sir Michal Cain, Johan Strauss, Telemann and Jerry Greenfield - you can say I'm in good company.
Cool.

Monday, February 12, 2007

House security flaw

My neighbor's alarm is working - drives us nuts. I went to see if they are connected to some security center, and saw 4 or 5 placards of the same company. I called them, gave the neighbors' full name and address (either would have been sufficient). They told me that the neighbors are not customers of theirs. So how come there are so many of your signs all around the house? - I asked. They answered that probably my neighbors installed their alarm system through the security company, but they are not active client of the phone service ("moked"). Hmmm, so basically, if I were a prospective burglar, I could call each of the phone services in the area giving them the details of the house I want to break into, and in a few minutes I would have a pretty good notion of the level of security I would have to deal with...

You may ask - why is it the service provider's fault? What should they do different? Well, for starts, they could say something like "we will check whether they are our customers, and if not we will contact the police.", leaving me no way to know which it will be.