Unison 2.40.61 binary for Nexenta Core Platform 3 (NCP3)

QUICK OVERVIEW

Nexenta Core Platform 3 is a Solaris-like operating system that has an OpenSolaris (RIP) kernel and GNU/Linux user-land commands layered on top. This gives the user the best of both worlds: A kernel that runs the amazing ZFS filesystem, with the familiar Linux toolset.

Unison is a file synchronization tool that is unmatched in it's safety and efficiency. It is optimized to run well over slow network links, using the rsync protocol to detect and send only the changed parts of files. It handles changes to files on both ends, including deletes making it a true two-way sync tool. In this way it has more in common with Dropbox than a backup tool. It is multi-platform, running on Linux, Windows, Macs, and Solaris.

Using Unison to sync multiple computers to a central ZFS server makes a great combination. Add ZFS snapshots and RAIDZ or mirroring of the sync'ed files makes it a great backup server as well.

UNISON 2.40.61 ON NCP 3

I am using Nexenta Core Platform 3, and I found that the apt repository has a very obsolete version of Unison (2.27.57). I have compiled Unison 2.40.61 for Nexenta Core 3.

Nexenta has an OCaml interpreter, but does not have the OCaml native compiler. I had to compile it to interpreted byte-code. To install it, first install OCaml: "apt-get install ocaml", then download the Unison binary from the link below and copy it to "/usr/bin".

Download the Unison 2.40.61 binary for Nexenta Core Platform 3 here.

This binary was compiled to interpreted byte-code, not native-code so although I have only tested it on Nexenta Core Platform 3, it may run on any platform that has an OCaml interpreter. If you try it on another platform please let me know and I will list it here.

Firefox on Windows 7: Downloads are disappearing

After upgrading to Windows 7 and installing Firefox I found that I couldn't download any files. I would download the files in Firefox and as soon as the download finished, the file would disappear. I tried turning off my anti-virus, I tried opening up my file permissions, ... nothing seemed to help.

Finally I found the solution. In Firefox to go the pseudo-URL "about:config", then scroll down to browser.download.manager.scanWhenDone and change it from 'true' to 'false' by double clicking it or right-click/modify.

That's it.
Hopefully I put enough keywords in this that people having the same problem will be able to find it, and I can save the next person a few hours of trial and error.

Upgrade your Inspiron 8600 to Windows 7 with this NVidia FX Go5200 video driver

I have an old Dell Inspiron 8600 I wanted to upgrade to Windows 7. The problem was that there is no current video driver for the Inspiron's Nvidia Geforce FX Go5200 video chip. Even the driver Dell is offering for download for Windows XP is the wrong one and causes all sorts of problems. There simply is no driver for Windows 7.

So I searched on NVIDIA to see what drivers might be close. I found these two possible candidates:

ForceWare 179.48 Win 7 driver for the Go7200
ForceWare 96.85 Vista driver for the FX5200 (NOT the FX Go5200)

I tried both, and I was able to get the ForceWare 96.85 driver working. Here's how...

I downloaded the ForceWare 96.85 driver from NVIDIA.
Link to driver

The device manager showed me that my Go5200 codes are: VEN_10DE&DEV_0324&SUBSYS_019C1028.
When I tried to install the ForceWare 96.85 driver (by running setup.exe) I was told it couldn't find a matching driver. I looked in the nv_disp.inf file and I saw lists of various NVIDIA video devices, but my Go5200 was not in the list. I simply added it to the list matching the format of the other lines.
(Note: if the install deletes the files after failing you may have to run the install and leave it waiting for you to click 'OK' while you copy the folder to another working folder)

I added lines in two places:

Where it had:

%NVIDIA_NV34.DEV_0322.1% = nv_NV3x, PCI\VEN_10DE&DEV_0322
%NVIDIA_NV34.DEV_0323.1% = nv_NV3x, PCI\VEN_10DE&DEV_0323
%NVIDIA_NV34.DEV_0326.1% = nv_NV3x, PCI\VEN_10DE&DEV_0326
%NVIDIA_NV34.DEV_0327.1% = nv_NV3x, PCI\VEN_10DE&DEV_0327

I added a line for my Go5200:

%NVIDIA_NV34.DEV_0322.1% = nv_NV3x, PCI\VEN_10DE&DEV_0322
%NVIDIA_NV34.DEV_0323.1% = nv_NV3x, PCI\VEN_10DE&DEV_0323
%NVIDIA_NV34.DEV_0324.1% = nv_NV3x, PCI\VEN_10DE&DEV_0324&SUBSYS_019c1028
%NVIDIA_NV34.DEV_0326.1% = nv_NV3x, PCI\VEN_10DE&DEV_0326
%NVIDIA_NV34.DEV_0327.1% = nv_NV3x, PCI\VEN_10DE&DEV_0327

Actually I added two lines, because when I Googled for DEV_0324 I saw some references to a different SUBSYS. Just in case I added both:

%NVIDIA_NV34.DEV_0322.1% = nv_NV3x, PCI\VEN_10DE&DEV_0322
%NVIDIA_NV34.DEV_0323.1% = nv_NV3x, PCI\VEN_10DE&DEV_0323
%NVIDIA_NV34.DEV_0324.1% = nv_NV3x, PCI\VEN_10DE&DEV_0324&SUBSYS_019c1028
%NVIDIA_NV34.DEV_0324.1% = nv_NV3x, PCI\VEN_10DE&DEV_0324&SUBSYS_01961028
%NVIDIA_NV34.DEV_0326.1% = nv_NV3x, PCI\VEN_10DE&DEV_0326
%NVIDIA_NV34.DEV_0327.1% = nv_NV3x, PCI\VEN_10DE&DEV_0327

And in one other place I changed this:

NVIDIA_NV34.DEV_0323.1 = "NVIDIA GeForce FX 5200LE"
NVIDIA_NV34.DEV_0326.1 = "NVIDIA GeForce FX 5500"

To this:

NVIDIA_NV34.DEV_0323.1 = "NVIDIA GeForce FX 5200LE"
NVIDIA_NV34.DEV_0324.1 = "NVIDIA GeForce FX Go5200"
NVIDIA_NV34.DEV_0324.3 = "NVIDIA GeForce FX Go5200 "
NVIDIA_NV34.DEV_0326.1 = "NVIDIA GeForce FX 5500"

Then I installed my new driver (setup.exe). I found my video chip was recognized! I was able to set it's proper native resolution in Control Panel/Display, and my "Performance" measurement went from 1.0 to 2.0. Not high enough for all of the Aero visual effects, but now I can watch video without any choppiness.

101 ZFS capable operating systems

With today's announcement of Oracle Solaris Express 11 our choices for running a ZFS capable Operating System just got even more confusing. I wasn't able to find a list of the build numbers these Operating Systems are based on all in one place, so I'm going to keep a chart of the choices here, and if readers will contribute in the comments I will keep the chart updated. The build number gives you a good idea of how recent the ZFS feature set is. If any of these become true forks with divergent code and feature sets we won't be able to rely on a single build number to identify the features any longer.

As of November 15, 2010:
Updated November 22, 2010 to include the KQ Infotech and Lawrence Livermore National Labs native ZFS on Linux:

OS	build	comments
Oracle Solaris Express 11 2010.11	b151a	licensed for testing only, encrypted fs
OpenSolaris 2009.06	b111b
OpenSolaris (last dev)	b134
OpenIndiana	b147
Nexenta Core 3.0.1	b134+	Linux userland, CLI only
NexentaStor Community	b134+	12TB limit, web admin
NexentaStor Enterprise	b134 +	not free, web admin
FreeBSD 8.1	pool 15	no CIFS or iSCSI
Linux fuse 0.6.9	pool 23	low efficiency
KQ ZFS	b121? pool 26	Native Linux port
LLNL ZFS	b147 pool 28	Native Linux port
Belenix 0.8b1	b111
Schillix 0.7.2	b147
StormOS "hail"		based on Nexenta
Jaris		Japanese
Milax 0.5	b128a	small size
Korona 4.5.0	b134	KDE
EON NAS	b130	embedded NAS

Please contribute info, links to distros, etc. and I will keep this page updated.
And yes, I know I didn't make it to 101 distros...
(Note: I have added this table to the ZFS Wikipedia article)

Tricky forensic hard drive acquisition of an iMac

Recently I had to do a forensic acquisition of an iMac hard drive, and it presented many obstacles.

The three basic ways to do an acquisition are:

1) Remove the hard drive and use high speed dedicated imaging hardware such as a Logicube, or from a forensic workstation through a write-blocker.

2) Boot the computer using a forensically sound Linux distro, such as Helix, and acquire the internal hard drive using a software tool such as dcfldd or LinEn to an external hard drive or over the network to a server set up for this purpose.

3) Boot the Mac into 'Target Disk Mode' (this only works on Macs) which exposes the internal hard drive on the Firewire port. Connect a forensic system to that firewire port through a write-blocker.

The iMac is notoriously difficult to disassemble because it uses a unibody enclosure, fitting all of the computer components inside the LCD monitor. Some models even have a tissue thin EMI foil wrapped around the edge of the LCD frame that would be difficult to remove and reattach without tearing. I always turn to iFixit.com to learn how to disassemble any computer or electronic device.

Because of this I would prefer not to use method #1, as removing the hard drive from this iMac would be time consuming and have the potential of damaging the system. I haven't damaged a computer yet, but I don't want this to be the first.

So I chose method #2, booting from a forensically sound Linux distro.

When booting a Mac into a forensic CD, never power up with the 'C' key to boot from the CD. If the system doesn't like your CD, for example if it is a version of Linux that won't boot on that particular system, it will bypass the CD and go right into booting from the hard drive. You will have just scribbled timestamps on the hard drive and your image wouldn't be forensically sound anymore. Instead power up with the 'OPTION' key to list what devices are bootable. Then you can choose the CD and if it won't boot it doesn't try to boot from the hard drive.

Also don't try booting with a wireless keyboard because if the battery has just run out the system won't see your 'OPTION' key and will boot into the hard drive: Scribble, scribble, not forensically sound anymore. Keep an extra wired apple keyboard around for this.

I started with Helix 2009R1 but it wouldn't boot properly. I next tried Helix 2008R1, Helix 1.9a, Helix 1.8, Backtrack (with no-mounting boot codes), DEFT 5.1, even an older Macquisition boot CD. No luck. All of them either wouldn't boot at all on the iMac or had a problem with the video. Apparently the new iMacs with the Intel Core i5 have a setting in the firmware that cause Linux to send all video to the mini display port instead of the LCD. My choice is now to either attach an external display to the mini display port or find a Forensic Linux CD that will work on this iMac.

Luckily I tried just one more, SystemRescueCD, and it worked. Be sure to add 'noswap' to the boot line so it doesn't try to mount any swap partitions it finds. I booted with the 'safest' choice, a VGA only video mode and it came up.

Now for the next challenge. A quick 'fdisk -l' shows that the system has a 2 Terabyte drive. Two Terabytes is the largest drive manufactured at this time. I prefer using DCFLDD, but I can't fit a 2 Terabyte image file plus overhead (file system, directories, etc) onto a 2 Terabyte drive. There are a few solutions to this. I could send the image over the network to a server. I could use a direct connect USB RAID appliance to get more than 2 TB of storage. But I had just read that Access Data has released a Linux command-line version of the excellent FTK Imager. I downloaded it and brought it over to the iMac on a USB thumb drive. I was able to image the internal drive using FTK Imager saving to Encase's E01 format with compression to an external 2TB drive so that the image files took up considerably less than 2TB.

I was so pleased with FTK Imager for Linux that I have replaced DCFLDD with FTK Imager for Linux from now on. It's an easy tool to use, it captures the drive serial numbers, the date and time, it gives good feedback, it's easier to use, and it saves to E01 format with compression.

Security or convenience?

So... I am sitting at a client's office waiting in-between meetings. The client has given me a desk where I can make some phone calls and do some work while I wait. I pick up the phone and call Cablevision (Optimum Online) to take care of something for my home residential cable service. Cablevision automatically recognizes the caller-ID number of the client I am calling from and drops me into a menu with full access to my client's account. I can make an automated payment from their bank account, I can check their balance due, their last payment date and amount, their billing history.

I hit a few keys until I can get a human operator. I tell him that the system has brought up the wrong account, I am calling about my residential account. I mention that their automated system gave me full access to someone else's account just because I used their phones. He tells me not to worry, they take security very seriously, and he will be asking me some security questions before allowing me access to any information.

"No," I tell him, "you've already given me full access to someone else's account. That is what I am telling you."

"Don't worry sir," he replies, "we have very strict security procedures and won't let that happen."

"You don't understand me. Your system has ALREADY given me access. Before I transferred to you it gave me full access to an account that isn't mine. It didn't ask me for any identification in any way."

"Yes sir, that is why I will be asking you a security question, to validate yourself to make sure no-one has access to another account's information."

"Never mind," I say in exasperation. This is going to be trouble for someone someday. But isn't it convenient?

ZFS powered file server in 15 minutes..

The OpenSolaris ZFS filesystem makes sophisticated features available inexpensively (free!) and makes them easy to use. Features such as deduplication, compression, checksums of all data, built-in RAID, snapshots, cloning, the best Windows CIFS server outside of Windows...

But forget all that for now. What many people need is a simple inexpensive reliable file server to hold their files, documents, photos, and media. What is the simplest way to set that up? I just built a ZFS file server for myself and I am going to walk you through how to do it.

ZFS can use regular SATA drives and prefers regular SATA connections, not expensive RAID cards. I used a Dell Optiplex 745 that I already had for this. It has two drive bays, and I will be putting two drives in and mirroring them. Of course you could use more drives and use RAID-Z instead of mirroring, but I want to stick to the simplest least expensive way to protect my data. Keep in mind that traditional RAID can only tell if the hard drive has hard errors, it does not keep checksums of all of your data and can not tell if you have data corruption. Studies have shown that with today's large disk drives bit errors are occurring more than you might think. ZFS can detect and correct these errors.

For the operating system I used Nexenta Core OS. Nexenta Core OS is a free version of Nexenta's larger offerings. Nexenta supports ZFS natively in an OpenSolaris based kernel with the more familiar Linux user-land commands. I used Nexenta instead of OpenSolaris because of the uncertainty around Oracle's acquisition of Sun and Solaris. Oracle has promised to continue developing Solaris, but will not be continuing development of the open-source version OpenSolaris. Nexenta on the other hand has promised continuing development of their OpenSolaris based OS.

I found installing Nexenta even easier than installing OpenSolaris. The entire install took about 15 minutes, most of that waiting for the disk to format and the operating system files to copy over. 15 minutes. The last time I installed Windows it took me hours.

I have boiled down the entire process to this simple checklist. For this example, we will make the assumptions that we want 2 mirrored drives, and a single networked share called "myshare".

1. Go to Nexenta.org and download the latest Nexenta OS. I chose Nexenta Core 3, which is the base operating system without any bells and whistles (not even a graphical interface). You could instead choose to go to NexentaStor.org and download the NexentaStor Community Edition, which adds a complete web based management system. You will download an ISO file and burn it to a CD.

2. Put two blank hard drives in your file server, and boot the Nexenta CD.

3. Answer the simple questions (Time Zone, root password, a non-root username, etc). Be ready to either use DHCP or have your IP address and DNS server info ready. Since this is a server, I want it to always have the same IP address. I entered IP 192.168.1.100, Netmask 255.255.255.0, gateway 192.168.1.1, and I used OpenDNS's DNS servers: 208.67.222.222 and 208.67.220.220. You could also use your ISP's DNS servers for possibly faster DNS. The install process will allow you to choose what disk(s) to install on. If you choose 2 or 3 disks it will set them up with Mirroring or RAID-Z.
After the install is done, let it shut down and remove the CD.

4. Boot into Nexenta and login as root, or as your non-root user and "sudo su -" to root. Type "zpool list" to see your root disk pool, which should be named "syspool". You now want to add a filesystem which will be your networked share. When you create the filesystem you will set all the options that make it work well with Windows and Macintosh instead of/in addition to Solaris:

zfs create -o utf8only=on -o normalization=formD -o casesensitivity=mixed -o nbmand=on -o aclinherit=passthrough syspool/myshare

5. Your filesystem is ready. Now you want to set where it will be visible on the server and where it will be visible to other computers on the network:

# make it visible at /myshare on the server itself
zfs set mountpoint=/myshare syspool/myshare
# enable file sharing
svcadm enable -r smb/server
# check /etc/pam.conf and make sure the following line is in it:
#         other password required pam_smb_passwd.so.1 nowarn
# create a user to own the files
useradd netuser
passwd netuser
# share the filesystem out to the network with the share name "myshare"
zfs set sharesmb=name=myshare syspool/myshare
# Check if it worked:
sharemgr show -vp

# its always a good idea on a new install to update apt, so later you won't have trouble installing things
apt-get update
apt-get upgrade

6. There are slight differences between the way Solaris handles file permissions (ACLs) and the way Windows handles permissions. After some aggravation I discovered it is best to manage permissions from Windows only:

# set permissions (ACLS) to be wide open, then go to a Windows computer and set the desired permissions

/usr/sun/bin/chmod -R A=everyone@:full_set:fd:allow /myshare

7. Now go to another computer and test it. Explore to your file server, "\\192.168.1.100\myshare" in my example. Better yet, go to tools/map network drive and give the share a drive letter. You should use the same username set up earlier, "netuser" in this example.

That's it! You're done.

Now in my case, I already had all of my data previously on this system running OpenSolaris. I pulled the original drive before installing Nexenta. Now that it is running I want to import all of my existing data, so I plugged in my old OpenSolaris drive and rebooted to get it recognized (since I can't figure out how to hot plug it) and issued these commands:

zpool import
# It lists all ZFS drives it sees, and I see my old one is called 'rpool'
zpool import -f -R /old rpool  # mount my old ZFS pool to /old
ls /old
cp -r /old/backups /myshare  # copy all the files over
cp -r /old/media /myshare

I kept this as simple as possible to allow anyone with minimal knowledge to set up an advanced ZFS file server in 15 minutes. There are other things you may want to add to this, of course, but this is enough to get you up and running. Some other topics I can address later are:

- You may want to allow different user names on your network to have their own files with separate permissions on the file server. You may also want to use a server name instead of the IP address 192.168.1.100.

- You may want a small boot/OS drive and larger redundant data drives.

- You may want to set up a script to take daily snapshots of your data. ZFS has the amazing ability to take a snapshot of a filesystem that only stores the data that has changed. There is no wasted disk space, so you can feel free to keep tons of snapshots at little cost. Some have snapshots taken every 15 minutes! You may also want a script to regularly run a "scrub" that checks for any disk errors and fixes them.

- If you chose Nexenta Core OS without a graphical interface, you may want to add a GUI:
apt-get install xorg
Xorg -configure
apt-get install gnome-core
apt-get install gdm # or just use 'startx' to manually start X-Windows

That's it for now. If I left out any steps please leave a comment and I'll add it.
.