How to build FreeNAS from Scratch

How to build FreeNAS from Scratch

Quickstart Guide for Developers and interested User

by Christian Degen aka bubulein [email protected]

Synopsis

After reading this Guide you will know how to Setup a FreeBSD System for compiling FreeNAS from Scratch. This Document will use the current FreeNAS Version based on FreeBSD 7.3. If you want to know more about FreeBSD 7.3 i refer to the FreeBSD 7.3-RELEASE Announcement

 

 

You need!

 

• a dedicated PC or Virtual Mashine
• a connection to the Internet
• 8GB Harddiskspace
• FreeBSD 7.3 Installation CD ( bootonly or disc1) (see FreeBSD 7.3-RELEASE Announcement)

 

 

 

It isn't necessary to use a dedicated box to compile FreeNAS. In this Guide we will setup our FreeNAS build environment inside a VirtualBox. This virtualisation software can be downloaded for free. Get VirtualBox. Virtualbox is available for Windows, Mac, Linux and OpenSolaris. ( also for FreeBSD

Also You can use VMware or Qemu.

 

 

Note on i386 and amd64
For compiling you need to setup a FreeBSD system according to the target architecture. This means if you like to build FreeNAS for amd64 you need to setup a FreeBSD amd64 system. ( same for i386). You can install an i386 system without problems on VirtualBox running on a amd64 Host.
If you want to setup FreeBSD amd64 on an i386 Host you have to use Qemu for emulating amd64. Please follow the Qemu Manual for doing this.

 

 

Please read the documentation of your Virtualisation Software for how to setup. Following a short explanation for VirtualBox.

VirtualBox

Installing VirtualBox

1. Get VirtualBox http://www.virtualbox.org/wiki/Downloads
2. Follow the Installation instruction acording to your Operatingsystem. Manual

Create a Virtual Machine for FreeBSD

1. Create “New” Virtual Maschine
2. Selecte the Name you want to use ( FreeBSD 7.3 - FreeNAS from Scratch)
3. Operating System: BSD - Version: FreeBSD
4. Select Memory you want to assign to your VirtualBox ( 256 MB suggested)
5. Create New Harddrive with atleast 8 GB of Diskspace

Installing FreeBSD 7.3

Now its time to download FreeBSD 7.3 if you have not already done that. There is no better Setup Guide for FreeBSD then the original one from the FreeBSD Handbook.

see: FreeBSD Handbook - Chapter 2 Installing FreeBSD

Preparing FreeBSD

After successful installation there are some steps needed to prepare the system.
In this example we install prebuild binaries Let'z Go!!

Connecting via SSH

The easiest way to work with the VirtualBox is connecting via SSH.
On Linux and Mac boxes your can use the Terminal to connect via SSH. On Windows you can use a SSH-Client like

Putty which can be downloaded here: PuTTY Download Page

If you don't know the IP of your FreeBSD system type

freenas-dev# ifconfig
em0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=8<VLAN_MTU>
        ether 00:e0:4c:d2:7e:ba
        inet 192.168.1.250 netmask 0xffffff00 broadcast 192.168.1.255
        media: Ethernet autoselect (100baseTX <full-duplex>)
        status: active
[...]

Now you can connect via ssh. Use the username you created during setup and the IP of your FreeBSD/Virtual Machine. To become the superuser root type

freenas-dev# su

 

 

Why I cannot su to root?

 

Be sure that your user is member of the group “wheel”. This group is able to become superuser. If not, following command will add your user to group wheel:

freenas-dev# pw usermod YOUR_USERNAME -G wheel

If you don't have a user run

freenas-dev# adduser

Be sure to assign the user to group “wheel”.

 

 

Updating FreeBSD

The first step is to update our FreeBSD system.
The command

freenas-dev# uname -a

shows you the current FreeBSD version. For example on my Virtual Maschine

FreeBSD freenas-dev.freenas.org 7.3-RELEASE FreeBSD 7.3-RELEASE #0: Sun Mar 21 06:15:01 UTC 2010     [email protected]:/usr/obj/usr/src/sys/GENERIC  i386

Start updating your FreeBSD with

freenas-dev# freebsd-update fetch install

This will fetch and install all updates.
After a reboot with

freenas-dev# reboot

The command

freenas-dev# uname -a

Should show you that you updated FreeBSD. see the -p1 in my example. This means we updated to “patchlevel 1”.

freenas-dev# uname -a
FreeBSD freenas-dev.freenas.org 7.3-RELEASE-p1 FreeBSD 7.3-RELEASE-p1 #0: Wed May 26 04:29:05 UTC 2010     [email protected]:/usr/obj/usr/src/sys/GENERIC  i386

Installing necessary applications

Following the commands you should execute to install the applications we need for compiling FreeNAS from scratch.

If you like to install them prebuild ( the easiest way) use:

freenas-dev# pkg_add -r subversion / 
pkg_add -r cdrtools / 
pkg_add -r portupgrade /  
pkg_add -r php5 / 
pkg_add -r perl / 
pkg_add -r ruby / 
pkg_add -r m4 / 
pkg_add -r bison / 
pkg_add -r xproto / 
pkg_add -r nano /
pkg_add -r bash / 
pkg_add -r cvsup-without-gui 

 

 

Depending on your shell execute “rehash” to publish new commands

 

 

Fetching the ports-tree

Now we fetch the ports-tree.
The ports-tree conatains all information we need to compile an application from source.

freenas-dev# portsnap fetch extract

This takes a few minutes. After fetching and extracting you find everything under /usr/ports. For more information about the ports-tree see Using the Ports Collection

If you later want to update the ports-tree you can use

freenas-dev# portsnap update

Also cvsup can be used - more on this topic later.

Downloading FreeBSD Sourcecode

For downloading the FreeBSD sources we use cvsup which we have already installed. Cvsup can also be used for updating the ports-tree. Do:

freenas-dev# cp /usr/share/examples/cvsup/ports-supfile /etc
freenas-dev# cp /usr/share/examples/cvsup/standard-supfile /etc/source-supfile

Then use your favorite editor ( nano, ee, vi) to change following lines in the files we copied above:

freenas-dev# ee /etc/source-supfile
from:
*default host=CHANGE_THIS.FreeBSD.org
to:
*default host=cvsup2.de.FreeBSD.org

see http://www.freebsd.org/doc/handbook/mirrors.html for a mirror next to you.

After editing the files run:

freenas-dev# cvsup -L 2 -g /etc/source-supfile
freenas-dev# cvsup -L 2 -g /etc/ports-supfile

for updating. ( ports-tree should be already uptodate)

Updating installed Ports

Earlier in this Guide we have installed the applications we need as prebuild binaries.
Sometimes these binaries are not 100% uptodate. To avoid this you can now run

portupgrade -av

This well use your ports-tree to compile this applications again with the most actual source from the ports-tree. While updating “portupgrade” ask you some options you ould use for building the applications from source. Its ok to use the default settings.

 

 

To avoid building X11 ( Graphical Desktop related) we create or edit /etc/make.conf and add the line
WITHOUT_X11=yes

 

 

 

 

Perhaps this the right time to create a Snapshot of your system. See The manual of your Virtual Machine for a “HowTo”
For VirtualBox see: First steps - Snapshots

 

 

Building FreeNAS

Setup FreeNAS build environment

Now we are going to setup the necessary directories and download the FreeNAS SVN.
Developers use Subversion to maintain current and historical versions of files such as source code, web pages, and documentation.
see: Apache Subversion on Wikipedia.org

Then i will guide you step by step through the interactive buildscript.

Getting FreeNAS from svn

Everything is build under /usr/local/freenas, so we create the directory and checkout the svn code.

freenas-dev# mkdir /usr/local/freenas 
 
freenas-dev# cd /usr/local/freenas/ 

freenas-dev# svn co https://freenas.svn.sourceforge.net/svnroot/freenas/branches/0.7 svn 

 

 

 

Error validating server certificate for 'https://freenas.svn.sourceforge.net:443':
 - The certificate is not issued by a trusted authority.//
 Use the fingerprint to validate the certificate manually!
Certificate information:
 - Hostname: *.svn.sourceforge.net
 - Valid: from Mon, 04 Jan 2010 14:21:55 GMT until Sat, 05 Feb 2011 09:03:23 GMT
 - Issuer: Equifax Secure Certificate Authority, Equifax, US
 - Fingerprint: ea:d1:3e:01:cc:16:e9:9b:c2:ab:4b:0c:cc:26:5f:25:78:ea:89:b4
(R)eject, accept (t)emporarily or accept (p)ermanently?

Here you can answer with (p), cause this isn't a error. You just don't have downloaded the certificate.

 

 

 

Compiling FreeNAS

To start the FreeNAS build script run

freenas-dev# /usr/local/freenas/svn/build/make.sh

The first screen appears and we can start

Welcome to the FreeNAS build environment.
Menu:
1  - Update the sources to CURRENT
2  - Build system from scratch
10 - Create 'Embedded' (IMG) file (rawrite to CF/USB/DD)
11 - Create 'LiveCD' (ISO) file
12 - Create 'LiveCD' (ISO) file without 'Embedded' file
13 - Create 'Full' (TGZ) update file
*  - Quit
>

For a first run you should go through all steps. Following i will guide you from here to your first FreeNAS LiveCD and embedded image.

Everything should be uptodate, but we do this step again just to illustrate later builds.

Select 1 - Update the sources to CURRENT

and if everything work as expected the answer should be

Checked out revision 5235.
=> Successful

Seclect 2 - Build system from scratch

Bulding system from scratch
Menu:
1 - Update source tree and ports collection
2 - Create filesystem structure
3 - Build kernel
4 - Build world
5 - Build ports
6 - Build bootloader
7 - Add necessary libraries
8 - Modify file permissions
* - Quit
>

Select 1 - Update source tree and ports collection

          ┌──────────────── FreeNAS - Update sources ────────────────┐
          │             Please select what to update.                │
          │ ┌──────────────────────────────────────────────────────┐ │
          │ │ [X] cvsup           Update source tree               │ │
          │ │ [X] freebsd-update  Fetch and install binary updates │ │
          │ │ [X] portsnap        Update ports collection          │ │
          │ └─────v(+)─────────────────────────────────────────────┘ │
          ├──────────────────────────────────────────────────────────┤
          │                  [  OK  ]       Cancel                   │
          └──────────────────────────────────────────────────────────┘

 

 

These steps we have already done, but it doesn't hurt you if you do it again!

 

 

Select 2 - Create filesystem structure

Should explain itself!

Select 3 - Build kernel

This will build the Kernel for your FreeNAS system.

  ┌──────────────────── FreeNAS - Build/Install kernel ─────────────────────┐
  │    Please select whether you want to build or install the kernel.       │
  │ ┌─────────────────────────────────────────────────────────────────────┐ │
  │ │               [X] prebuild  Apply kernel patches                    │ │
  │ │               [X] build     Build kernel                            │ │
  │ │               [X] install   Install kernel + modules                │ │
  │ └─────────────────────────────────────────────────────────────────────┘ │
  ├─────────────────────────────────────────────────────────────────────────┤
  │                         [  OK  ]       Cancel                           │
  └─────────────────────────────────────────────────────────────────────────┘



  ┌─────────────────────── FreeNAS - Kernel patches ────────────────────────┐
  │ Select the patches you want to add. Make sure you have clean/origin     │
  │ kernel sources (via cvsup) to apply patches successful.                 │
  │ ┌─────────────────────────────────────────────────────────────────────┐ │
  │ │         [X] 3ware   3ware serial ATA RAID controller driver         │ │
  │ │         [X] a100u2  A100U2 U2W-SCSI-Controller                      │ │
  │ │         [X] aac     Adaptec Raid Controller driver                  │ │
  │ │         [X] ata     Misc. ATA driver patches                        │ │
  │ │         [X] iscsi   iSCSI initiator                                 │ │
  │ │         [X] nkpt    Patch number of kernel page tables              │ │
  │ │         [X] wol     WOL for nVidia(nfe(4)) and 3Com(xl(4))          │ │
  │ │                                                                     │ │
  │ │                                                                     │ │
  │ │                                                                     │ │
  │ │                                                                     │ │
  │ │                                                                     │ │
  │ │                                                                     │ │
  │ │                                                                     │ │
  │ └─────────────────────────────────────────────────────────────────────┘ │
  ├─────────────────────────────────────────────────────────────────────────┤
  │                         [  OK  ]       Cancel                           │
  └─────────────────────────────────────────────────────────────────────────┘

If unsure keep all settings as is. This will compile the same Kernel who is released with the official FreeNAS builds!
Now your System need some time to build the Kernel.

Now you are back on the previous screen, and we start building the world

Select 4 - Building World

This will copy all necessary files from the build system the your FreeNAS environment.

Now we start building the Ports.

Select 5 - Build ports

And select “build”.
Then you can select which ports you like to build. Select “ALL” for a standard FreeNAS build.

                 ┌────── FreeNAS - Build/Install Ports ──────┐
                 │ Please select whether you want to build   │
                 │ or install ports.                         │
                 │ ┌───────────────────────────────────────┐ │
                 │ │        build    Build ports           │ │
                 │ │        install  Install ports         │ │
                 │ └───────────────────────────────────────┘ │
                 ├───────────────────────────────────────────┤
                 │          [  OK  ]       Cancel            │
                 └───────────────────────────────────────────┘


  ┌──────────────────────────── FreeNAS - Ports ────────────────────────────┐
  │                 Select the ports you want to process.                   │
  │ ┌─────────────────────────────────────────────────────────────────────┐ │
  │ │[X] AA_geomraid5      GEOM RAID5 filesystem                          │ │
  │ │[X] arcconf           Adaptec SCSI RAID administration tool          │ │
  │ │[X] ataidle           Utility to set spindown timeout for ATA drives │ │
  │ │[X] bash              The GNU Project's Bourne Again SHell           │ │
  │ │[X] bsnmp-ucd         A bsnmpd module impl. parts of UCD-SNMP-MIB    │ │
  │ │[X] bsnmptools        SNMP client-side tools                         │ │
  │ │[X] cdialog           An enhanced version of 'dialog'                │ │
  │ │[X] clog              Circular log file tool                         │ │
  │ │[X] e2fsprogs         Utilities to manipulate ext2/3/4 filesystems   │ │
  │ │[X] fdisk             PC slice table maintenance utility (fixed)     │ │
  │ │[X] firefly           Multithread daapd Apple iTunes server          │ │
  │ │[X] fuppes            Free UPnP Entertainment Service                │ │
  │ │[X] fusefs-ntfs       Mount NTFS partitions and disk images          │ │
  │ │[X] geom_eli          GEOM Eli filesystem encryption                 │ │
  │ └────v(+)─────────────────────────────────────────────────────────────┘ │
  ├─────────────────────────────────────────────────────────────────────────┤
  │                         [  OK  ]       Cancel                           │
  └─────────────────────────────────────────────────────────────────────────┘

Select 5 - Build ports

Select this option again, but now choose “install” instead of “build”

  install  Install ports   

This will install all your prebuild ports along with needed files from you base FreeBSD system.

Select 6 - Build bootloader

To build and install the bootloader.

Select 7 - Add necessary libraries

Should be self explained.

Select 8 - Modify file permissions

The last step will set the correct permissions for you FreeNAS files.

Now quit with * and you are back on the previous screen.

Welcome to the FreeNAS build environment.
Menu:
1  - Update the sources to CURRENT
2  - Build system from scratch
10 - Create 'Embedded' (IMG) file (rawrite to CF/USB/DD)
11 - Create 'LiveCD' (ISO) file
12 - Create 'LiveCD' (ISO) file without 'Embedded' file
13 - Create 'Full' (TGZ) update file
*  - Quit

Now the last step in our build process. Choose one of the option from 10 - 13 to create the final FreeNAS ISO-Image or embedded file.

In this example i choose option 11 for creating a LiveCD.

Thats all

After a successful build you find your Ready to Use FreeNAS ISO and/or IMG in /usr/local/freenas

[root@harry /]# ls /usr/local/freenas
FreeNAS-i386-0.7.2.5234.checksum        builds
FreeNAS-i386-LiveCD-0.7.2.5234.iso      rootfs
FreeNAS-i386-embedded-0.7.2.5234.img    svn
bootloader                              work

Hopefully this document helps you to build your own FreeNAS from scratch.

If you have questions feel free to send me an email, or the preferred way is to ask your question on the FreeNAS forum.
FreeNAS forum on Sourceforge