How Samba was written

Methods: 1, documents; 2, sniff; 3, protocol scanner; 4, differential technique


  




http://www.samba.org/ftp/tridge/misc/french_cafe.txt



How Samba was written

---------------------



Andrew Tridgell

August 2003





Method 1:

---------



First off, there are a number of publicly available documents on the

CIFS/SMB protocol. The documents are incomplete and in places rather

inaccurate, but they are a very useful starting point. Perhaps the

most useful document is "draft-leach-cifs-v1-spec-02.txt" from 1997

which is a protocol specification released by SNIA and authored

primarily by Microsoft (with significant input from many other people,

including myself). This document has expired as an IETF draft, and

Microsoft has dropped their attempts to get CIFS accepted as an IETF

standard, but the document is still available if you look hard enough

with an internet search engine.



There are numerous other public specifications for various pieces of

the protocol available. I maintain a collection of the ones I know

about in http://samba.org/ftp/samba/specs/



Method 2:

---------



I call this method the "French Cafe technique". Imagine you wanted to

learn French, and there were no books, courses etc available to teach

you. You might decide to learn by flying to France and sitting in a

French Cafe and just listening to the conversations around you. You

take copious notes on what the customers say to the waiter and what

food arrives. That way you eventually learn the words for "bread",

"coffee" etc. 



We use the same technique to learn about protocol additions that

Microsoft makes. We use a network sniffer to listen in on

conversations between Microsoft clients and servers and over time we

learn the "words" for "file size", "datestamp" as we observe what is

sent for each query.



Now one problem with the "French Cafe" technique is that you can only

learn words that the customers use. What if you want to learn other

words? Say for example you want to learn to swear in French? You would

try ordering something at the cafe, then stepping on the waiters toe

or poking him in the eye when he gives you your order. As you are

being kicked out you take copious notes on the words he uses.



The equivalent of "swear words" in a network protocol are "error

packets". When implementing Samba we need to know how to respond to

error conditions. To work this out we write a program that

deliberately accesses a file that doesn't exist, or uses a buffer that

is too small or accesses a file we don't own. Then we watch what error

code is returned for each condition, and take notes. 



Method 3:

--------



Method 3 is a greatly expanded variant of the "swear words" technique

I have already mentioned. It involves writing something called a

"protocol scanner". A protocol scanner is a program that tries all

possible "words" in some section of a protocol and uses the response

to automatically deduce new information about the protocol. It is like

the French Cafe technique but with a very patient waiter.



For example, some section of the protocol might contain a 16 bit

"command word" that tells the server what operation to perform. There

are 64 thousand possible command words, so we try all of them and note

which ones give an error code other than "not implemented". Then we

need to work out how much supplementary data each command word needs,

so the program tries 1 byte of blank data, then 2 bytes then 3 bytes

etc until the server changes its response in some way. When the

response changes then you know (with a fairly high level of confidence

at least) that you are using the right quantity of data. You then try

using non-blank data, putting in a filename or a directory name or a

username until the server changes its response again. After a large

number of tries the program eventually finds a combination of data

that gives no error code at all - the server has accepted our request!

We have just discovered a new phrase in "French".



Once the server has accepted the new request we need to work out what

the request actually does. We know its a valid command, but what does

it do? To determine that we send the new command then we follow it up

with a series of already understood commands that ask the server for

lots of detailed information about the files it has. Has a file size

changed?  Has a date changed? Has a file changed its name? Eventually

we work out what the command does.



Method 4:

--------



The final method that is worth describing here is the "differential"

technique. This is used to discover interactions between different

command words. Using the (now rather stretched) French Cafe analogy it

is like trying to work out if you should use a different word for

coffee if you are having it with a biscuit than if you are having it

with cake. It goes like this.



You use your new knowledge of French to write a virtual waiter. A

program that is supposed to behave like a real French waiter. Then you

write another program that sends a random series of French phrases in

turn to the real waiter and your virtual waiter. Your program then

examines the replies carefully and notes any differences in how the

two waiters respond. You keep careful notes.



When the two waiters respond differently then you look at your notes

and try the same sequence of phrases again, but this time leaving one

of them out. Do the two waiters now behave in the same way? If they do

then you know that phrase is critical to the difference between the

two waiters, otherwise it isn't. In this way you can quickly determine

the minimum set of phases that causes the two waiters to respond

differently.



Once you have this minimal set then you stare at it hard and use the

methods described earlier to see whats wrong with your virtual

waiter. When you fix it you try again, and keep trying until your

waiter behaves the same as the virtual waiter.



Now imagine using all of the above techniques (plus some other similar

techniques I have not gone into here) over a period of 12 years. Thats

how Samba was written.