OmniOS: Installation and Basic Setup

From the CD/DVD to the working system, no extras

This article is part of a series focused on the building and setup of a home-NAS based on ZFS and OmniOS. The index is located here: An OmniOS ZFS Fileserver.

First of all, this article must be considered always "in progress" because I will update it in the future whenever I find a better configuration or better setup commands, especially for the file permissions. As result, you can check the history of the article and the date of the latest revision down at the end.

Installation and user setup

At this point I had two SSD disks installed together, but I not yet set as hardware mirror.

I installed OmniOS on the first SSD using the CD/DVD and an external USB2 DVD-RW drive. Once installed I logged in using the virtual KVM provided by iLO3 (a terrible experience: without license it stops working every 30-60 seconds).
Once logged in I added the network interfaces, since they are not set automatically during installation:

$ dladm show-phys
$ ipadm create-if e1000g0
$ ipadm show-if
$ ipadm create-addr -T static -a 10.0.0.5/8 e1000g0/v4
$ route -p add default 10.0.0.1
$ echo 'nameserver 8.8.8.8' >> /etc/resolv.conf
$ echo 'nameserver 8.8.4.4' >> /etc/resolv.conf
$ cp /etc/nsswitch.dns /etc/nsswitch.conf

As DNS I provided the one offered by Google, since they are very fast. A second and equivalent alternative are the ones provided by OpenDNS, also very fast. I didn't bother setting up IPv6, it's not needed on my local network (for the time being).

Since I had already set up my ZFS data disks using OpenIndiana I simply imported them using:

$ zfs import

and then I recreated users and groups as I had in the old system as explained further below.

In case I had no disks, I would have created the ZFS datasets this way:

$ format -e < /dev/null
$ man zpool
$ zpool create -O casesensitivity=mixed -O utf8only=on -O normalization=formD -O nbmand=off tank mirror vdev0a vdev0b mirror vdev1a vdev1b
$ zfs set compression=lz4 tank

where "format -e" provides a list of disk device names available on the system (stop immediately with Control-C after seeing the list), "man zpool" is useful to check whether my commands are correct (I cannot verify them right now), then the next command "zpool create ..." creates the pool named "tank" out of 4 hard disks: vdev0a and vdev0b are mirrored, vdev1a and vdev1b are mirrored and the two mirrors are striped. As result, a RAID10 is created.
Concerning the options for zpool, check by yourself. Some of them won't be editable later and have to be set now, some others can be edited later, but they will be inherited and the firther commands are simplified.
I decided to use "nbmand=off" because, from what I understood, it is useful when both AFP and SMB are being used. In my case I'm using only SMB at the time being.
I activated "compression=lz4" because it offers some compression and better performances than uncompressed data and it simply has no real disadvantages.

In my case I had the two SSD, but the pool was already there thanks to the installation and I had to add the second disk. I followed the instructions from How to Set Up a ZFS Root Pool Mirror in Oracle Solaris 11 Express. These the steps but check them accurately before executing them, in case I have typos.

$ cfgadm -s "select=type(disk)"
Ap_Id                          Type         Receptacle   Occupant     Condition
sata0/0::dsk/c14t0d0p0         disk         connected    configured   ok
sata0/1::dsk/c14t1d0p0         disk         connected    configured   ok
$ zpool status rpool
  pool: rpool
 state: ONLINE
  scan: none requested
config:

        NAME        STATE     READ WRITE CKSUM
        rpool       ONLINE       0     0     0
          c14t0d0p0 ONLINE       0     0     0

$ fdisk -W - c14t0d0p0
$ fdisk -W - c14t1d0p0
$ fdisk -A 191:128:199:8:0:254:63:1023:12544:104855296 c14t1d0p0

$ prtvtoc /dev/rdsk/c14t0d0s0 | fmthard -s - /dev/rdsk/c14t1d0s0

$ zpool attach -f rpool c14t0d0s0 c14t1d0s0
$ zpool status

$ installgrub /boot/grub/stage1 /boot/grub/stage2 /dev/rdsk/c14t1d0s0

The first step is to check device names to be used later; the second step checks which one is already in use; the third step applies the Solaris partition configuration to the second disk; the fourth step copies SMI labels (I got some warnings here); the fifth step adds the second disk to the pool as mirror; the last step installs the bootloader on the second disk too.
A similar procedure can be found on OpenSolaris ZFS rpool mirror and OpenIndiana - How to mirror the root disk.

Define a strategy for the folder permissions

I used again the structure suggested by http://breden.org.uk/2009/05/10/home-fileserver-zfs-file-systems/, that means common folders are placed in the root of the pool and are represented by a separate dataset, and all the users' homes are located in "/tank/home". Each shared folder belongs to root and to a different group, so that I can define read/write permissions by adding and removing the users to the different groups. The media folders have a more complex setup, described later.

The commands used to create the datasets are:

$ zfs create -o aclinherit=passthrough-x -o aclmode=passthrough tank/home
$ zfs create -o aclinherit=passthrough-x -o aclmode=passthrough tank/home/olaf
$ zfs create -o aclinherit=passthrough-x -o aclmode=discard tank/media
$ zfs create -o aclinherit=passthrough-x -o aclmode=passthrough tank/media/Music
$ zfs create -o aclinherit=passthrough-x -o aclmode=passthrough tank/Temporary

I then proceeded with the creation of the groups:

$ groupadd -g 501 olaf
$ groupadd -g 509 music
$ groupadd -g 505 temps
$ groupadd -g 512 musicr

$ chgrp olaf /tank/home/olaf
$ chgrp music /tank/media/Music
$ chgrp temps /tank/Temporary

$ chmod g+rwx /tank/home/olaf
$ chmod g+rwx /tank/Temporary
$ chmod g+rwx /tank/media/Music
$ chmod g+s /tank/media/Music

and I finally added a real user:

$ mkdir /export/home/olaf
$ zfs create rpool/export/home/olaf
$ useradd -u 501 -g olaf -s /bin/bash -d /export/home/olaf -c "Olaf Marzocchi" -m olaf
$ usermod -G music,temps olaf
$ chown olaf:olaf /export/home/olaf
$ chown olaf /tank/home/olaf

And then, in order to be able to use the "pfexec" command that gives specific permissions (administrator or root, in my case) without the need of issuing "su" or "sudo", I issued:

$ usermod -P 'Primary Administrator' olaf

At this point I only needed SMB shares, so I pasted "other password required pam_smb_passwd.so.1 nowarn" at the end of pam.conf and I executed:

$ passwd olaf

to be able to use SMB to share data: SMB uses a different password so I had to reset the user password. What is left is the settings for the specific shares. On each dataset I executed:

$ zfs set sharesmb=on dataset

to effectively enable sharing. In case a different share name is desired, the line becomes:

$ zfs set sharesmb=name=desired_share_name dataset

At this point I updated the system and rebooted:

$ pkg update -nv
$ pkg update
$ reboot

Concerning the media folders (Music) I wanted a more exotic configuration:

  • some users must be able to read and add files (the users that correspond to a real person)
  • some users must only have read rights, no permission to add, delete or modify files (these users are the media players on tablets and televisions)
  • my user should have full power
  • everyone else should not have any access.

The solution is to use (non-trivial) ACLs and for this goal I created the musicr user group: read-only.
To apply the permissions, I executed:

$ chmod -R A=user:olaf:full_set:fd:allow,owner@:full_set: fd:allow, group@:read_set/add_file/add_subdirectory/execute /synchronize:fd:allow, group:musicr:read_set/execute/synchronize:fd:allow, everyone@:full_set:fd:deny /tank/media/Music

This way the restricted users have only read rights (read_set/execute/synchronize:fd:allow), normal users can create and read files (read_set/add_file /add_subdirectory/execute /synchronize:fd:allow), the file owner and I have full power (full_set:fd:allow), while everyone else has nothing (full_set:fd:deny). Since only the owner has the right to delete files, I avoid the risk of having someone deletes other people's data.
Important note: the synchronize attribute should not be implemented, but it is translated to the NTFS equivalent when the file is shared via CIFS and has the following effect (from ntfs.com): "Allows or denies different threads to wait on the handle for the file or folder and synchronize with another thread that may signal it. This permission applies only to multithreaded, multiprocessing programs".
I tested it myself and, depending on the application (MX Player, Oplayer, MplayerX, WD TV Live) and platform (OS X, Android, iOS, WD TV Live) the file could or could not be played/read. With the attribute set, no problems at all.

I also created the WD TV Live user:

$ mkdir /var/WDTVlive
$ groupadd -g 511 wdtvlive
$ less /etc/group
$ useradd -g 511 -G musicr,temps -u 511 -d /var/WDTVlive -s /bin/false wdtvlive
$ passwd wdtvlive

Of course, do not apply these instructions to production machines without checking each command line by yourself.

If you have questions or comments, please drop me an email. I will answer and eventually update this page.

Further information are also available on the OmniOS Admin Guide

Update (2015-10-10)

With the previous configuration, the home folder did not have any ACL set beyond the standard ones (there is a specific term for these, I forgot it), and that always worked fine because I was using Netatalk, that relies on UNIX permissions.

Since I switched to SMB (kernel CIFS) for file sharing, ACL were needed: SMB users appear as "user[user]:[permissions]", and SMB groups appear as "group:[numerical code]:[permissions]". I added the required ACLs and I set them to be inherited, but then dovecot was not able to properly access mail, located in /tank/home/olaf/Maildir. The issue was specifically with lock files, and I was getting a lot of errors like these about "rename" and "unlink" and permissions denied.

dovecot: [ID 583609 mail.error] imap(olaf): Error: rename(/tank/home/olaf/Maildir/.Mailing Lists.Social.Facebook/dovecot-uidlist.tmp, /tank/home/olaf/Maildir/.Mailing Lists.Social.Facebook/dovecot-uidlist) failed: Permission denied
dovecot: [ID 583609 mail.error] imap(olaf): Error: unlink(/tank/home/olaf/Maildir/.Mailing Lists.Social.Facebook/dovecot-uidlist.tmp) failed: Permission denied

At the end I made another ZFS dataset with standard settings and permissions and now it works again.

$ zfs create -o compression=on tank/mail
$ chgrp mail /tank/mail
$ mkdir /tank/mail/olaf
$ mv /tank/home/olaf/Maildir /tank/mail/olaf/
$ chown -R olaf:olaf /tank/mail/olaf
$ find Maildir -type d -exec chmod 700 {} \;
$ find Maildir -type f -exec chmod 600 {} \;
$ svcadm enable dovecot

Update (2016-06-12)

With OmniOS r151018 SMBv2 has become available. This gives much better performances and lower CPU requirements on the clients, so I installed it.

I found that the server did not appear anymore in the network servers shown by Windows, and that the addresses ".local" like "OmniOS.local:9091" (that would be the TransmissionBT client) also didn't work anymore. The source of the problem is the lack of NetBios in the default configuration of the new release. It can be solved with

$ pfexec sharectl set -p netbios_enable=true smb

I also found that some folders could not be read anymore. It turned out it was my mistake due to a misunderstanding of the way ACLs work.

The solution was to remove "everyone@:full_set:fd:deny" from the set of permissions granted to the folders.

Update 2018-03-31

During a complete reinstall of OmniOS, with datasets ready and so on, the creation of the user, after the groups are ready, was performed like this:

zfs create rpool/export
zfs set mountpoint=/export rpool/export
zfs create rpool/export/home
useradd -u 501 -g olaf -s /bin/bash -d /export/home/olaf -c "Olaf Marzocchi" -m olaf

Author: Olaf Marzocchi

First revision: 2013-09-24.
Last revision: 2018-04-02.