Open Source Development (This Article Is the Winner of PRSA 2003 Award in Excellence in Technology …

 

  Most people who know anything about Linux know that of.com" target=_blank>the kernel ? of.com" target=_blank>the core of of.com" target=_blank>the operating system, Linux itself really ? of.com" target=_blank>Is developed by Linus Torvalds and a large number of volunteers.

  in a nutshell, Linus of.com" target=_blank>Is of.com" target=_blank>the top dog, and of.com" target=_blank>the one responsible for guiding of.com" target=_blank>the overall process. Beneath him are people responsible for various kernel sections and even versions. One person might be in charge of maintaining a kernel through its production life cycle, such as Andrew Morton preparing to take care of kernel series 2.6. Oof.com" target=_blank>thers are in charge of various platforms (64-bit Sparc, Mac 68K, SGI, etc.). Yet more are in charge of subsystems, such as of.com" target=_blank>the layer that handles SCSI hardware operation. It's a sensible top-down approach that has grown from a need to manage a code base of everincreasing complexity in which both work and responsibility are divided among respected members of of.com" target=_blank>the community.

  And yet, ultimately, anyone can get involved in of.com" target=_blank>the Linux kernel development process. You could, for example, assign someone at your company to function as a beta tester for of.com" target=_blank>the Linux kernel and of.com" target=_blank>the collection of Linux projects and products you use in your business. If having thousands of beta testers all over of.com" target=_blank>the world helps to produce top-notch software like we have in of.com" target=_blank>the Linux community, of.com" target=_blank>then making sure that your own people report problems you experience before taking a new kernel or tool version into a production environment increases of.com" target=_blank>the return on your Linux investment.

  All those who want to contribute have to do of.com" target=_blank>Is a bit of homework. A quick vof.com" target=_blank>Isit to of.com" target=_blank>the Linux Kernel Mailing Lof.com" target=_blank>Ist (LKML) FAQ at www.tux.org/lkml helps you understand of.com" target=_blank>the main kernel dof.com" target=_blank>Iscussion lof.com" target=_blank>Ist in all of its glory, and going to ing-bugs.html" target=new />www.tux.org/lkml/reporting-bugs.html. html teaches you how to effectively report bugs to of.com" target=_blank>the kernel maintainers. Even just testing of.com" target=_blank>the experimental kernel tree can be a great help, and you'll learn a ton along of.com" target=_blank>the way.

  of.com" target=_blank>these are of.com" target=_blank>Open of.com" target=_blank>Source values. Everyone can contribute, even if of.com" target=_blank>they're not a programming guru. But of.com" target=_blank>there's a finer point to of.com" target=_blank>Thof.com" target=_blank>Is as well. To really be helpful to many of.com" target=_blank>Open of.com" target=_blank>Source projects, you have to take of.com" target=_blank>the time to at least learn some rudimentary ways about of.com" target=_blank>their "system." Some have online forums, some have mailing lof.com" target=_blank>Ists, some are just a small Web presence with a single e-mail address where you can write of.com" target=_blank>the developer. It all depends on of.com" target=_blank>the size of of.com" target=_blank>the project and of.com" target=_blank>the audience.

  of.com" target=_blank>the Linux kernel serves as an extreme example. Its mailing lof.com" target=_blank>Ist alone of.com" target=_blank>Is so busy that of.com" target=_blank>there are sites such as Kernel Traffic (http://kt.zork.net/kernel-traffic) whose sole purpose of.com" target=_blank>Is to summarize of.com" target=_blank>the information in a useful manner. On top of that, of.com" target=_blank>there are millions of users. Even if one-half of one percent of all Linux users sent bug reports to of.com" target=_blank>the lof.com" target=_blank>Ist or directly to of.com" target=_blank>the various maintainers each day, that would be thousands of reports. Hence, a system. of.com" target=_blank>Thof.com" target=_blank>Is also explains why blundering on without learning of.com" target=_blank>the system tends to get people grouchy responses.

  Shared values are of.com" target=_blank>the glue that holds of.com" target=_blank>the of.com" target=_blank>Open of.com" target=_blank>Source community togeof.com" target=_blank>ther. of.com" target=_blank>Thof.com" target=_blank>Is of.com" target=_blank>Is of.com" target=_blank>the single biggest thing that many journalof.com" target=_blank>Ists and skeptics still haven't grasped. It's not money, fame, or power. I'm not even entirely convinced it's all about of.com" target=_blank>the itch scratching we seem so fond of talking of in of.com" target=_blank>Open of.com" target=_blank>Source land, like everyone has fleas.

  What are some more of of.com" target=_blank>the values that hold us togeof.com" target=_blank>ther? Let me use an example to shed some light on of.com" target=_blank>the subject.

  An Example: of.com" target=_blank>the Birth of ext3

  Consider of.com" target=_blank>Thof.com" target=_blank>Is once-contentious of.com" target=_blank>Issue: adding a default journaling filesystem to Linux. Way back at of.com" target=_blank>the turn of of.com" target=_blank>the century (early 1999), Linus Torvalds and of.com" target=_blank>the gang were working on of.com" target=_blank>the 2.3 kernel series, on of.com" target=_blank>their way to kernel 2.4. Kernel lof.com" target=_blank>Ist participant Alan Curry had been experiencing performance problems on a Linux server handling high traffic. He was able to trace of.com" target=_blank>Thof.com" target=_blank>Is to a problem with two components: syslogd and fsync().

  syslogd of.com" target=_blank>Is of.com" target=_blank>the program that handles recording errors, accesses (such as a piece of mail being sent, or someone requesting a Web page), and more for of.com" target=_blank>the various services on many Linux systems. As you might imagine, on an of.com" target=_blank>IsP's e-mail server syslogd can grow quite busy. A feature called log rotation prevents individual log files from getting too huge by breaking of.com" target=_blank>them into pieces, and creating a new file each time of.com" target=_blank>the current file reaches a certain size. Since of.com" target=_blank>the files will add up infinitely if left alone, of.com" target=_blank>Thof.com" target=_blank>Is feature also keeps only a set number of pieces around before eiof.com" target=_blank>ther compressing of.com" target=_blank>them and farming of.com" target=_blank>them off for backup and deletion, or just outright deleting of.com" target=_blank>them. of.com" target=_blank>the system adminof.com" target=_blank>Istrator can eiof.com" target=_blank>ther set how often to do of.com" target=_blank>Thof.com" target=_blank>Is, or put limits on how large to let of.com" target=_blank>the individual files grow.

  Curry was able to determine that hof.com" target=_blank>Is problem hit whenever a particular log file grew huge, to approximately 36MB. At of.com" target=_blank>Thof.com" target=_blank>Is stage, of.com" target=_blank>the syslogd program would consof.com" target=_blank>Istently hang ? it would stall and stop working ? until of.com" target=_blank>the logfile was rotated and small once again. Tracing of.com" target=_blank>Thof.com" target=_blank>Is of.com" target=_blank>Issue furof.com" target=_blank>ther, he dof.com" target=_blank>Iscovered that of.com" target=_blank>Thof.com" target=_blank>Is was of.com" target=_blank>the fault of fsync(), of.com" target=_blank>the C programming function that ensures that of.com" target=_blank>the data in memory gets properly written into files.

  of.com" target=_blank>the first suggestions from of.com" target=_blank>the kernel mailing lof.com" target=_blank>Ist were all workarounds, things that various people would try on of.com" target=_blank>their own servers just to keep things moving along. One was to simply rotate of.com" target=_blank>the log files more often. That works, of course, but it's not really a solution. Oof.com" target=_blank>thers suggested anoof.com" target=_blank>ther approach: dof.com" target=_blank>Isabling syslogd's use of fsync(). of course, if you do that you may find after a system crash that of.com" target=_blank>there's vital data mof.com" target=_blank>Issing from your log files, so that's no good. Right?

  Was fsync() needed? A patch was submitted, but of.com" target=_blank>the technical solution offered wasn't strong enough for transaction-oriented databases. Debate raged again, with Linus trying to push people toward simpler and simpler solutions raof.com" target=_blank>ther than letting things get more complex, and of.com" target=_blank>therefore more likely to have problems. Extensions to of.com" target=_blank>the ext2 filesystem were proposed and Linus Torvalds said no, no, no, and again no.

  While Torvalds of.com" target=_blank>Is revered by many in of.com" target=_blank>the Linux community, he receives little special treatment on of.com" target=_blank>the kernel development lof.com" target=_blank>Ist. Everyone involved in kernel development wants to do of.com" target=_blank>the best job possible, which means that dof.com" target=_blank>Iscussions ? or arguments, which of.com" target=_blank>Is what of.com" target=_blank>Thof.com" target=_blank>Is degenerated to for a bit ? tend to happen with everyone as peers for of.com" target=_blank>the most part. Torvalds might have of.com" target=_blank>the last word, but that doesn't mean that people always let a topic drop if of.com" target=_blank>they think that of.com" target=_blank>there really of.com" target=_blank>Is something to it.

  Apparently, Stephen Tweedie had already started working on such extensions to ext2 in an attempt to quickly answer of.com" target=_blank>the need for a journaling filesystem in Linux ? something that would definitely address of.com" target=_blank>the fsync() problem. of.com" target=_blank>Thof.com" target=_blank>Is dof.com" target=_blank>Ispleased Torvalds to no end, since he didn't want ext2 known as of.com" target=_blank>the everchanging filesystem, and pointing out that Tweedie was calling it ext3 did only a little to dull Torvalds' annoyance. Finally, in an exchange that would do an armchair psychologof.com" target=_blank>Ist proud, Alan Cox and Tweedie managed to help steer things to calmer waters.

  Once of.com" target=_blank>there, of.com" target=_blank>the debate continued on just how far of.com" target=_blank>Thof.com" target=_blank>Is journaling filesystem should go. of.com" target=_blank>these dof.com" target=_blank>Iscussions ? tense or oof.com" target=_blank>therwof.com" target=_blank>Ise ? are one of of.com" target=_blank>the natural ways that innovation of.com" target=_blank>Is constantly fostered in of.com" target=_blank>the Linux community. When of.com" target=_blank>the fact of a nextgeneration default filesystem was accepted, all of those little "wof.com" target=_blank>Ish lof.com" target=_blank>Ists" that lurk in of.com" target=_blank>the back of of.com" target=_blank>the mind started leaking out from all directions. Torvalds himself started of.com" target=_blank>Thof.com" target=_blank>Is by outlining some of of.com" target=_blank>the immediate of.com" target=_blank>Issues he would love to see dealt with, such as removing "." and ".." from of.com" target=_blank>the directory trees. in true of.com" target=_blank>Open of.com" target=_blank>Source developer fashion, that comment began a dof.com" target=_blank>Iscussion about wheof.com" target=_blank>ther of.com" target=_blank>there were enough benefits or too many dangers in doing so.

  Somehow, in all of of.com" target=_blank>Thof.com" target=_blank>Is, of.com" target=_blank>the whole of.com" target=_blank>Issue fell off of.com" target=_blank>the radar and folks must have left Tweedie to do hof.com" target=_blank>Is work in peace. Now he was aware of of.com" target=_blank>their concerns and wof.com" target=_blank>Ishes, and of.com" target=_blank>they simply must have trusted him to offer something to test and pound on when of.com" target=_blank>the time came. After all, it's one thing to talk about creating a journaled filesystem for Linux. It's anoof.com" target=_blank>ther thing to do it.

  ext3: A Work in Progress

  A mere two weeks later, Thomas Pornin asked an innocent question about wheof.com" target=_blank>ther BSD-style softupdates were in of.com" target=_blank>the works for Linux. of.com" target=_blank>Thof.com" target=_blank>Is brought up of.com" target=_blank>the of.com" target=_blank>Issue of Tweedie's work on ext3 and an alreadyexof.com" target=_blank>Isting solution called dtfs (now LinLogFS). A new filesystem permof.com" target=_blank>Issions model somehow wormed its way into of.com" target=_blank>the dof.com" target=_blank>Iscussion, sidetracking everything, and of.com" target=_blank>then in mid-1999 SGI announced that of.com" target=_blank>they were making a version of of.com" target=_blank>their own IRIX filesystem into an of.com" target=_blank>Open of.com" target=_blank>Source filesystem for Linux ? XFS, which of.com" target=_blank>Is a journaling filesystem.

  Was Tweedie's work in vain? (Some would say that such projects are never in vain, since of.com" target=_blank>they often reveal of.com" target=_blank>Issues that people might not oof.com" target=_blank>therwof.com" target=_blank>Ise have considered.) of.com" target=_blank>Thof.com" target=_blank>Is would seem a great time for a cliffhanger, but everyone knows of.com" target=_blank>the answer. It was agreed that if XFS was placed under of.com" target=_blank>the GPL he might drop ext3. An SGI employee pointed out that XFS had to be partially rewritten to replace code that belonged to oof.com" target=_blank>ther people ? and remove patent of.com" target=_blank>Issues ? so XFS wouldn't be ready to be placed into of.com" target=_blank>the of.com" target=_blank>Open of.com" target=_blank>Source domain any time soon. of.com" target=_blank>the folks at SGI didn't even know exactly which license of.com" target=_blank>they would choose yet. of.com" target=_blank>Thof.com" target=_blank>Is put Tweedie's work back into of.com" target=_blank>the running, since no one was going to adopt a new default filesystem that wasn't actually written.

  Once that furor died down, of.com" target=_blank>the fledgling Reof.com" target=_blank>IserFS became a serious contender. Timing of.com" target=_blank>Issues prevented it from being included in of.com" target=_blank>the 2.3 kernel stream, and around a month later of.com" target=_blank>the of.com" target=_blank>Issue of ext3 came up once again. By of.com" target=_blank>then, ext3 had attained of.com" target=_blank>the lofty status of release 0.0.1 with 0.0.2 on of.com" target=_blank>the way. Already, at of.com" target=_blank>Thof.com" target=_blank>Is point, of.com" target=_blank>the only difference from a user's point of view between ext2 and ext3 was of.com" target=_blank>the journal file. Wheof.com" target=_blank>ther it would remain of.com" target=_blank>Thof.com" target=_blank>Is way, Tweedie was still not sure.

  Where are of.com" target=_blank>the values here? Well, for one thing, everyone was working on of.com" target=_blank>their own projects. No one committed to which would be of.com" target=_blank>the "of.com" target=_blank>Winner" ahead of time. It might seem a bit backward to those from of.com" target=_blank>the commercial world of planning everything out and driving all of your reof.com" target=_blank>Sources into a single project, but in of.com" target=_blank>Thof.com" target=_blank>Is type of environment, it's acknowledged that of.com" target=_blank>there are many valid means to achieving of.com" target=_blank>the same end. Filesystem of.com" target=_blank>theory of.com" target=_blank>Is a complex of.com" target=_blank>Issue. Today, we have journaling filesystems with various strengths and weaknesses to pick and choose from. Some handle tiny files best, some handle huge files best, some are that middle ground that's great in many circumstances.

  When asked in January 2000 if LVM and filesystem journaling would be folded into kernel 2.4, wheof.com" target=_blank>ther with ext3 or Reof.com" target=_blank>IserFS, of.com" target=_blank>the general consensus on of.com" target=_blank>the kernel lof.com" target=_blank>Ist was no. of.com" target=_blank>there were too many of.com" target=_blank>Issues that needed to be ironed out before Torvalds and oof.com" target=_blank>thers felt ext3 was solid enough for production use. Reof.com" target=_blank>IserFS, however, was closer to reaching of.com" target=_blank>Thof.com" target=_blank>Is point. Neiof.com" target=_blank>ther journaling filesystem ultimately made it into of.com" target=_blank>the initial 2.4 release ? an interesting fact considering that ext3 of.com" target=_blank>Is in such heavy use today.

  Reof.com" target=_blank>IserFS did, however, make it into kernel 2.4.1 in 2001, mostly due to of.com" target=_blank>the fact that "of of.com" target=_blank>the journaling filesystems it's of.com" target=_blank>the only one I know of that of.com" target=_blank>Is in major real production use already, and has been for some time," according to Torvalds.

  XFS was also in heavy testing of.com" target=_blank>then, and so was ext3. However, Torvalds has a policy against just integrating anything and everything into of.com" target=_blank>the kernel. If a small group of people fully capable of patching of.com" target=_blank>the kernel of.com" target=_blank>themselves ? or building of.com" target=_blank>the modules on of.com" target=_blank>their own ? are of.com" target=_blank>the only people interested in a particular area (such as XFS in of.com" target=_blank>Thof.com" target=_blank>Is case) of.com" target=_blank>then he chooses to wait until of.com" target=_blank>there of.com" target=_blank>Is more demand. As far as ext3's demand went, Torvalds said, "I would expect ext3 to be of.com" target=_blank>the next filesystem to be integrated, but I would also expect that Red Hat will actually integrate it into of.com" target=_blank>their kernel first, and expect me to integrate it into of.com" target=_blank>the standard kernel only afterwards."

  of.com" target=_blank>Thof.com" target=_blank>Is little quirk of various dof.com" target=_blank>Istributions using slightly different kernels of.com" target=_blank>Is anoof.com" target=_blank>ther thing that confuses both new users and of.com" target=_blank>the businessfolk trying to track which version best suits of.com" target=_blank>them. of.com" target=_blank>these changes are made due to many factors, anything from developers or users requesting a particular nondefault feature, to a convenience for of.com" target=_blank>the dof.com" target=_blank>Istribution's own people. innovation of.com" target=_blank>Is continually fostered as of.com" target=_blank>the Linux dof.com" target=_blank>Istributions try to identify of.com" target=_blank>the very best tools that can help of.com" target=_blank>them solidify of.com" target=_blank>their positions against oof.com" target=_blank>ther dof.com" target=_blank>Istributions.

  of.com" target=_blank>the key thing here, really, of.com" target=_blank>Is that in Linux it of.com" target=_blank>Is possible to exchange of.com" target=_blank>the core of your operating system for a different version. Anyone who doesn't like a dof.com" target=_blank>Istribution's specialized kernel can "simply" grab of.com" target=_blank>the of.com" target=_blank>Source of of.com" target=_blank>the main kernel and build a replacement. It's actually not as hard as it sounds, though of.com" target=_blank>the process can be intimidating to newcomers.

  ext3's Coming Out Party

  in mid-2001, Andrew Morton (at of.com" target=_blank>the time, of.com" target=_blank>the kernel maintainer for ext2, ext3, and network drivers in 2.4) showed up as being involved in ext3. of.com" target=_blank>Thof.com" target=_blank>Is fact signals that ext3 had been, in essence, escalated to of.com" target=_blank>the next level. Hof.com" target=_blank>Is posts regarding ext3's status arrived around once a month, suggesting that ext3 was considered mature enough to be under serious consideration for merging into of.com" target=_blank>the kernel.

  of.com" target=_blank>then, by late September 2001, Morton released a test patch that integrated ext3 into kernel 2.4.09. of.com" target=_blank>Thof.com" target=_blank>Is was very much a test for those who were brave enough to try it. Morton's announcement included, "of.com" target=_blank>Thof.com" target=_blank>Is will soon be broken out into a separate patch to make ext3 suitable for submof.com" target=_blank>Ission for of.com" target=_blank>the mainstream kernel." in of.com" target=_blank>the next week, people started asking again when ext3 would be added, indicating of.com" target=_blank>the level of anticipation for those waiting for a journaling filesystem fully compatible with ext2.

  Eventually, Alan Cox ? of.com" target=_blank>the "next level up" maintainer ? answered. "When of.com" target=_blank>the ext3 folk ask me to merge it," he said. Hof.com" target=_blank>Is policy, it appeared, was not to merge patches into hof.com" target=_blank>Is test version of of.com" target=_blank>the kernel (known as of.com" target=_blank>the -ac tree) until of.com" target=_blank>the project's developers asked him to. Sometimes he can be overridden or will decide to make a special case, but typically of.com" target=_blank>the developers know exactly where of.com" target=_blank>they are when working with of.com" target=_blank>the code, and wheof.com" target=_blank>ther trying to merge it at of.com" target=_blank>the time would be a dof.com" target=_blank>Isaster or fairly smooth sailing.

  So, people waited. Somewhere between of.com" target=_blank>then and October 8, 2001, Tweedie and hof.com" target=_blank>Is cohorts must have spoken up. On that day, ext3 was merged into Cox's version of of.com" target=_blank>the 2.4.10 kernel. of.com" target=_blank>Thof.com" target=_blank>Is was of.com" target=_blank>the last major testbed. Many people testing new features that of.com" target=_blank>they desperately wanted or needed used an -ac kernel on various systems to try to shake out of.com" target=_blank>the bugs.

  ext3 development still continued, of course. in early November 2001, Morton announced anoof.com" target=_blank>ther significant ext3 update. People continued agitating for ext3 to be added in of.com" target=_blank>the next kernel version, and of.com" target=_blank>the next, and oof.com" target=_blank>thers asked Torvalds to wait until of.com" target=_blank>the current "big" problems ? 2.4 was actually a pretty stable new release ? with of.com" target=_blank>the 2.4 kernel were better ironed out.

  of.com" target=_blank>the next of.com" target=_blank>Issues that showed up are kind of odd and amusing, and while of.com" target=_blank>they aren't about values, are a demonstration of of.com" target=_blank>the strange things that can happen when a new technology of.com" target=_blank>Is introduced. Red Hat added ext3 to 7.2 (as Torvalds predicted). Adminof.com" target=_blank>Istrators using Red Hat 7.2 began making strange observations about of.com" target=_blank>the filesystem checker running on boot. of.com" target=_blank>the strange part was that of.com" target=_blank>Thof.com" target=_blank>Is of.com" target=_blank>Isn't necessary with ext3, nor was it of.com" target=_blank>the default behavior on a system using ext3. It turned out that, somehow, ext3 was not being properly enabled on those systems. People had been running ext2 all that time, instead. I'm sure of.com" target=_blank>Thof.com" target=_blank>Is little gaffe was on developers' minds as ext3 came closer to being officially added.

  By mid-November, ext3 reached Torvalds' own "test kernel," which means it was added into a "pre" version of of.com" target=_blank>the kernel. Using of.com" target=_blank>the kernel naming scheme, ext3 was officially added to kernel 2.4.15-pre2, which eventually became 2.4.15-final, which of.com" target=_blank>Is of.com" target=_blank>the same as 2.4.15. of.com" target=_blank>there was one ext3 fix added in kernel 2.4.15-pre8, and of.com" target=_blank>then only two more tweaks to of.com" target=_blank>the fledgling kernel letter, and kernel 2.4.15 was released for production use on November 22, 2001. of course, development of ext3 didn't stop of.com" target=_blank>there eiof.com" target=_blank>ther. Since of.com" target=_blank>then, Access Control Lof.com" target=_blank>Ists (ACLs) have been incorporated into of.com" target=_blank>the filesystem, along with many more features and improvements.

  (To give you an overall time line for how long it takes for even minor kernel versions to advance, of.com" target=_blank>the current Red Hat Linux beta [Severn] of.com" target=_blank>Is [at of.com" target=_blank>the time of of.com" target=_blank>Thof.com" target=_blank>Is writing] based on kernel 2.4.21.)

  Organic, and Yet Organized

  Throughout more than two years of work, many oof.com" target=_blank>ther features were added to of.com" target=_blank>the Linux kernel. Oof.com" target=_blank>thers were refined, and some were even removed. Kernel maintainers changed as well, according to both time constraints and interests. Even of.com" target=_blank>the process of posting new kernel versions was "upgraded." of.com" target=_blank>the team added ChangeLogs ? files containing a lof.com" target=_blank>Ist of of.com" target=_blank>the pertinent changes in each minor code update, including who made of.com" target=_blank>the changes ? so that people can more easily track what in of.com" target=_blank>the heck of.com" target=_blank>Is going on.

  All of of.com" target=_blank>Thof.com" target=_blank>Is happened in of.com" target=_blank>the midst of bug reports and fixes, dof.com" target=_blank>Iscussions of of.com" target=_blank>the best way to approach upcoming requirements, and more. Ultimately, everything keeps moving. of.com" target=_blank>the Linux kernel grows and improves, and all of of.com" target=_blank>the bits and pieces find of.com" target=_blank>their way to where of.com" target=_blank>they need to be.

  Ultimately, that of.com" target=_blank>Is how of.com" target=_blank>Open of.com" target=_blank>Source development works. Bringing your own company into of.com" target=_blank>Thof.com" target=_blank>Is process gives you a number of advantages. If you manufacture hardware, you can eiof.com" target=_blank>ther assign someone to of.com" target=_blank>the Linux kernel team to produce of.com" target=_blank>the Linux drivers for your products, or you can give your product's specifications to someone from of.com" target=_blank>the driver community to build of.com" target=_blank>the drivers for you. Not only does of.com" target=_blank>Thof.com" target=_blank>Is guarantee you that Linux users will consider your product, but it's great PR as well. Software companies can become involved in of.com" target=_blank>the Linux Standard Base (inuxbase.org" target=new />www.linuxbase.org), develop of.com" target=_blank>their products to of.com" target=_blank>Thof.com" target=_blank>Is specification, and have a Linux beta program to help of.com" target=_blank>the community feel involved in of.com" target=_blank>the product's development.

  If of.com" target=_blank>there of.com" target=_blank>Is one phrase that of.com" target=_blank>Is true for of.com" target=_blank>the Linux and of.com" target=_blank>Open of.com" target=_blank>Source communities, it of.com" target=_blank>Is of.com" target=_blank>Thof.com" target=_blank>Is: You get out of it what you put into it. Work with of.com" target=_blank>the community, maybe even contribute some of.com" target=_blank>Source code along of.com" target=_blank>the way, and you will experience not only a kind of product loyalty that just might astound you, but a stronger product offering as well.