NFS Help

I like to think that I’m a fairly intelligent guy, able to pick up most things reasonably quickly given the opportunity.  After all, I transitioned from a Windows-only SE into an SE with a good reputation for VMware ESX Server, various Linux flavors, Mac OS X, and some Cisco configuration (hey, if you can do GRE tunnels with IPSec encryption, you’re not too shabby with IOS).  But I’m having a real problem with NFS.

I know, it seems silly, but I just can’t wrap my head around how it works.  In particular, the NetApp implementation of NFS and the /etc/exports file that Data ONTAP uses seems to be very different than the way you would configure NFS on Linux or Solaris.  Even when I go through the FilerView GUI to configure an NFS export, it doesn’t seem to work the way I expect.  To be fair, I’m sure this is just a lack of understanding on my part and not necessarily a flaw or drawback in the NetApp implementation.

Take this example.  I recently added another old F840 storage system to my lab at the office, and will begin setting up a demo SnapMirror environment to show to customers (SnapMirror with VMware on NetApp is going to be slick!).  I thought I’d also start performing some NFS testing as well; I’m particularly interested in thin provisioning the VMDKs on a thin provisioned FlexVol via NFS.  So I create a new FlexVol and then proceed to configure a new NFS export.  After walking through the NFS export wizard in FilerView and specifying my MacBook Pro’s IP address as having both read/write access and root access, I mount the export and proceed to try to copy an ISO image file.  Denied!  Huh?  Checking the properties, I see that I only have read-only permissions.  What’s up with that?

I try several other variations, and all of them provide the same result.  How can that be?  If my host’s IP address is provided read/write access, why do I have read-only access?  Is one option overriding another?  How do the options interact with each other?  I’m sure these are silly/easy questions for those well-versed in NFS, but for whatever reason I’m having a hard time here.

If anyone could share some enlightening information, I’d certainly appreciate it.

Tags: , , , , , , ,

Me again! (Haven’t yet followed up with the last post.)

One of the fun things I have ran into with Solaris->ONTAP is that if you do not have Kerberos configured (default!) and do the basic “mount foohost:/barpath /bar”, Solaris mounts with NFSv4. Since you don’t have Kerberos setup, you get read-only permissions no matter what you do.

Telling Solaris to use NFSv3 fixes the issue completely. Now, I’m commenting from a Mac but I don’t have it on the same subnet as my filer, so I can’t test to see if this issue is present under OS X as well as Solaris.

Hope this helps,

A.

What version of ONTAP are you running? Here’s a line from our 8xx series:

/vol/vol1 -sec=sys,ro=10.250.38.0/24,rw=10.250.38.5,root=10.250.38.5,nosuid

Substitute the 10.250.38.5 with your mac IP. Try this with and without the root=xxx clause.

Also do a “showmount -e $filer_ip” from your mac and see what gets returned.

Adam,

It’s my understanding that OS X does not support Kerberized NFS or NFS v4, although I could be mistaken. I’ll try obtaining a Kerberos ticket from the realm to which the filer is joined (an AD domain) and see if that makes any difference.

Jimmie,

I’m running ONTAP 7.1.2.1 (the latest release supported on the F8xx series; 7.2 isn’t supported). I’ll try that /etc/exports line and see if it helps at all. Thanks, and keep the suggestions coming!

Why don’t you show us the line from /exports for the share. Also are you using the finder to do the mount or the command line. Try the command line if you used the finder and show the command here. A showmount -e as jimmy says would be good.

Paste the exports line as suggested above…. but also, make your client use NFSv3 to make troubleshooting easier. Paste the output of “mount” so we can see if it got mounted read-only or got mounted read-write but doesn’t allow you to write. On top of things, if you are root, root has special permissions and you need to make sure root squash is not off. If it is, root becomes the user nobody or nobody4 depending on your OS.

Snapmirror is the bomb by the way. Oh, and Netapp is the bomb too.

Sorry, that should be make sure root squash is not on. (but read the security implications first). Also, forgot to mention if you change permission stuff, you should exportfs -f from the filer to make sure the acl’s get flushed. I always use command line for the Netapp stuff, not sure if the GUI has a -f option for re-exporting.

an addendum to my previous post: we’re running nfs.v3 over tcp. God bless brotha.

All,

I’ll post the exports line when I am next back in the office; sorry for the delay. I am not running as root (root account is disabled by default in OS X), and I should be using NFS v3 by default. Thanks for the suggestions–keep ‘em coming!

OK, here’s the /etc/exports line that is currently in effect:

/vol/fvol_vmdata -sec=sys,ro,rw=10.16.2.149,root=10.16.2.149

(IP addresses have been changed to protect the innocent, but I’ve verified that they are correct for the host that should be connecting.)

I just tried again after obtaining a Kerberos ticket from the AD domain of which the storage system is a member (and options nfs.kerberos is on), but I still get read-only access. Argh!

Given that you have sec=sys Kerberos is not in the picture. They are sec=krb5 etc.

Are you sure the requests are coming from the ip address you think they are, has it got more than one interface etc ?

What does showmount -e give.

What does trying the mount from the command line do.

Try setting option nfs.mountd.trace on and examine /etc/messages.

What does exportfs -c 10.16.2.149 /vol/fvol_vmdata rw sys

If you can mount the volume, it’s likely not the export line. I’d bet its simply “non-matching” UIDs, aka a permission problem. The export is with “sys” security (the default), which means standard UNIX permissions are in effect. So the client says “I’m uid=500″ and the server trusts that, and checks to see if that uid has permission on the local filesystem. So you have to matchup UIDs on both sides. My guess is that the directory you’re trying to write to isn’t owned by the UID of your account on the MBP.

As root on the MBP chown the directory on the exported volume to your OS X uid, and you’ll be in business.

Also, drop the “ro” option from your export. “ro” and “rw” next to each other is confusing. :)

James,

I had tried the export several times, some of them with Kerberos security specified, hence the referral to Kerberos and Kerberos tickets. But you’re right, as the export is listed right now, there is no mention of Kerberos.

“showmount -e” on my MacBook Pro reports an error; I don’t know how to run “showmount” on the NetApp storage system (is there an equivalent command in Data ONTAP?).

I do have nfs.mountd.trace on, but hadn’t checked /etc/messages–doh! (I was expecting the trace output to be logged to the console.) I’ll have a look and see what’s listed there, if anything.

Mark,

That was my suspicion as well, but I’m not sure how it is I’m supposed to get around that. Root is disabled on my MBP (and I run as a non-administrative user), and AFAIK there’s no way to set ownership within Data ONTAP. And wouldn’t granting my MBP root access take care of that problem? I suppose I could su to my administrative account and then chown the mounted NFS volume, but it still seems like I’m missing something.

showmount -e showing an error is interesting :)

The exportfs -c 10.16.2.149 /vol/fvol_vmdata rw sys should show the effective permissions.

From a random manual as I am at home and away from my filer and I haven’t brought a simulator up on my mac.

“-c
Checks whether an NFS client has a specific type of access to a file system path. You must specify the IP address of the NFS client (hostip) and the exported (not actual) file system path (path). To check whether the NFS client has read-only, read-write, or root access to the file system path, specify the ro, rw, or root option, respectively. If you do not specify an access type, Data ONTAP simply checks whether the NFS client can mount the file system path. If you specify an access type, you can also specify the NFS client’s security type: sys, none, krb5, krb5i, or krb5p. If you do not specify a security type, Data ONTAP assumes the NFS client’s security type is sys. Note: If Data ONTAP does not find an entry in the access cache corresponding to (1) the file system path and (2) the NFS client’s IP address, access type, and security type, Data ONTAP (1) determines the NFS client’s host name from its IP address (for example, it performs a reverse DNS lookup), (2) checks the NFS client’s host name, access type, and security type against the file system path’s export options, and (3) adds the result to the access cache as a new entry.”

To become root on a mac do the following as a admin user

simonsmachine:/Users/jb3134 dirk$ sudo bash
Password:
simonsmachine:/Users/jb3134 root#

you can use su to go from non admin to admin user.

The question here is is the filesystem being mounted read/write or do you not have permission to write the files ?

the root= just says not to map the root user to nobody, so you can do privileged operation over nfs. The account doing the operation still needs permission to do the operation. So the first question is is the mount read-write or read-only ?

James,

I do have an admin account to which I can switch using su, but it’s not root and I prefer not to enable root on my MBP. I haven’t tried the “exportfs -c” command yet, but when I get back into the office next week I’ll try that and see what I get. I am able to mount the NFS export, but I do not have read-write access from my MBP.

I’ll try the “exportfs -c” command and double-check the contents of /etc/messages on the storage system to see if either of those command shed some additional light on the problem.

Thanks for the help!

Scott,

It not that the root account isn’t enabled you just need to jump though hoops to run processes as root.

ps -auxww | grep root |wc -l

Will show a number of processes running as root, 52 on my mac as this instant.

>> OK, here’s the /etc/exports line that is currently in effect:
>> /vol/fvol_vmdata -sec=sys,ro,rw=10.16.2.149,root=10.16.2.149

try this:
/vol/fvol_vmdata -sec=sys,rw=10.16.2.149,root=10.16.2.149,ro

Best regards.

Another note:
NFS is the best you can get for VMware datastores.
Tons of benefits, and almost no cons (Especially with WAFL from NetApp of ZFS from SUN).
For example see this discussion thread:
http://www.vmware.com/community/thread.jspa?threadID=51698&start=0&tstart=0

best regards.

Another thing to try:
/vol/fvol_vmdata -sec=sys,rw=10.16.2.149,root=10.16.2.149,ro,anon=0

best regards.

Scott I have this exact problem with my MB Pro and our new netapp and my DataONTAP sim. did you every find a solution for this problem?

its drivin me nuts..

thanks,

ScottMcD,

I did get it fixed. I believe–and I could be wrong–that the issue had something to do with the fact that the Qtree at the root of the volume was set to Mixed security style and needed to be changed to either UNIX or NTFS. I can’t recall which way I changed it; I’ll check when I get in the office tomorrow and post another comment with the information.

Great thanks Scott,

i did a qtree from the command line and my volume /vol/mrkt is set to unix. also can you tell me how you mounted the nfs volume on your mb, command, Finder, etc..

Thanks,

ScottMcD

I found that when I looked at my qtree as well and changed it to unix, that the same problem you had described above resolved itself.

I think with your MAC, you have to either do the work as root which is a common uid of 1 and ensure that root=mac’s ip, you need a common set of uid/gid to do the work on these net apps

The exportfs -c really helped me, it returned the following (note, this is when my qtree was set to mixed)

mofs011> exportfs -c 172.xxx.xxx.xxx /vol/hds rw sys
exportfs: 172.xxx.xxx.xxx does not have rw access to /vol/hds (No access cache entry, try again)
exportfs: 172.xxx.xxx.xxx has rw access to /vol/hds
mofs011> exportfs -c 172.xxx.xxx.xxx /vol/hds root sys
exportfs: 172.xxx.xxx.xxx does not have root access to /vol/hds (No access cache entry, try again)
exportfs: 172.xxx.xxx.xxx has root access to /vol/hds

I reran it again now and I get:
exportfs: 172.xxx.xxx.xxx has root access to /vol/hds

I did everything except looking at the Qtree type. This Blog directed me in the Proper direction. What a fool I was!!!

Thanks a million