blog.scottlowe.org

In UNIX/Linux integration scenarios, it’s useful to know which accounts have been UNIX-enabled, i.e., have had the UID number, NIS domain, login shell, and home directory attributes configured.

It’s certainly very possible to do this with command-line tools such as AdFind or DsQuery, but users may also find it useful to have a saved query available within the Active Directory Users & Computers console for easy reference.

The way to do this is define a custom query using this string:

(objectCategory=Person)(objectClass=User)(uidNumber=*)

If you add just this text and nothing else in the “Find Custom Search” dialog box (the Advanced tab), then the console will automatically add ampersands and additional parentheses to turn it into a “proper” LDAP query that will show you any account that has a UID number configured. Certainly, additional fields like loginShell or unixHomeDirectory could be added as well, but this query will probably be sufficient for most instances.

I started not to publish this, but figured if I couldn’t remember the exact syntax then someone else might not be able to remember the syntax either. This one is as much for me as it is for others.

Reader Scott Merrill pointed out something to me in an e-mail regarding a Registry change that might be necessary in some Active Directory integration scenarios:

Finally, I would like to share one registry change that we’ve found to be necessary in our AD integration. By default, the MS LDAP server only returns 1,000 results. As a university department with more than 1000 active students, this limitation has caused us some frustration.

This KB article shows how to increase the number of results returned in a query: http://support.microsoft.com/kb/315071

We recently set MaxPageSize to 5,000. I don’t know if this will
introduce additional problems down the road, but at least it lets me fully enumerate all our AD users from a Linux machine with `getent passwd`.

If you have an Active Directory domain with more than 1,000 users in the DN specified in your LDAP configuration, then this is a Registry change you’ll want to investigate. Otherwise, you could find that your UNIX/Linux servers aren’t able to fully enumerate all the users in the domain.

Thanks, Scott!

I’m posting this stuff here on the off chance that it someday might be useful to someone out there somewhere. About four years ago, I had a wild hunch to start learning Novell NetWare 6.5, and to perform some integration testing with some other technologies with which I was already familiar. Along the way, I gathered these notes. I make no warranties about the accuracy, validity, or relevance of this information; I’m just publishing it here in case it may prove useful later. (You never know.)

So, that being said, here are the notes:

• SSH “shell” access to NetWare 6.5 server: The SSHD NetWare Loadable Module (NLM) had to be loaded first. Attempts to login failed; the sshd_config file had to be edited and a Novell-specific directive (eDirNameContext) had to be modified in order to add the context where the admin account was stored (in this case, OU=Users.O=Company). After the configuration file was modified and the SSHD NLM unloaded and loaded again (to reflect the changes to the configuration file), logins via SSH were successful. (Note: It appears that NetWare 6.5 does not support the Blowfish-CBC cipher.)
• SFTP access to NetWare 6.5 server: After successful SSH “shell” access (see previous bullet point), SFTP access also worked correctly. Tests using Fugu (a native Mac OS X SFTP application) were successful and without any major events or problems. In fact, SFTP was used to transfer the files necessary for the VNC testing (see next bullet point) to the NetWare server.
• VNC access to NetWare 6.5 console: Using SFTP, a VNC server NLM was copied to the server. After setting the VNC password (using VNCPASS.NLM) and loading the VNC server (VNCSRV.NLM), access to the NetWare server’s GUI via VNC was successful. The VNC client used was Chicken of the VNC, a freeware Mac OS X VNC client. Performance was on par for LAN access to a server.
• Native file access from Mac OS X: As indicated in several online sources, the AFPTCP NLM had to be unloaded and then reloaded with the CLEARTEXT option. Then the SYS volume on the server could be mounted using the Go To Server command. After an initial login, the AFPTCP NLM was unloaded and reloaded without the CLEARTEXT option, and everything continued to work just fine.
• Rconsole access from Mac OS X: Using RconJ, a Java-based port of Rconsole to Mac OS X, Rconsole access was successful. The RCONAG6 NLM had to be loaded first on the server in order for this to work.
• VNC inside SSH tunnel: Creating SSH tunnels (using the –L switch) works in NetWare 6.5 just as it does with Linux or OpenBSD. Using the VNC NLM discussed earlier and an SSH tunnel, the VNC traffic was secured and encrypted across the wire. This worked exactly as expected.
• Native file access from Windows XP: Initial attempts to access the server from a Windows XP system failed (authentication problems). The NDS user object had been created in iManager and a simple password had also been created in iManager as well (necessary before CIFS will work). However, the cifsctxs.cfg file (that specifies contexts) had not been updated with the correct context (OU=Users.O=Company, which is where all user objects are stored). After modifying this file and reloading CIFS, then access from Windows XP still failed (network path not found). Further tests showed that typing the UNC path from the Run command on the Start menu failed, but browsing through My Network Places or typing the UNC path including a share name worked just fine.
• NTP on NetWare 6.5: XNTPD.NLM is an NTP daemon for NetWare, similar in implementation and purpose as NTP on Linux or OpenBSD. Upon editing the NTP.CONF file in SYS:\ETC, XNTPD could be loaded only after TIMESYNC.NLM was unloaded. Even then, XNTPD seemed to unload occasionally and without reason, and the NTPDATE utility had to be used to manually synchronize the time.
• Autoloading specific NLMs on startup: Upon reboot, the VNC, SSH, and Rconsole NLMs weren’t loaded, and so the server was inaccessible except from the console. Using the “rconag6 encrypt” command, a LDRCONAG.NCF file was created with an encrypted Rconsole password. Then, AUTOEXEC.NCF was edited to reference this file (in order to load the Rconsole agent) as well as the SSH and VNC NLMs. This would ensure that the necessary NLMs were loaded every time the server booted.
• Universal passwords: After some difficulty mounting a volume from Mac OS X, setting passwords, and such, the server was rebooted and Universal Passwords were enabled for the Users.Company container. The passwords were then set for various accounts. Following that, native file access from both Mac OS X and Windows XP (with one caveat; see below) worked flawlessly. The caveat for Windows XP native file access is that browsing shares using just the server name in the UNC path does not work; at least one share name must also be included (i.e., \vsninteg does not work, but \vsninteg\sys works just fine). SSH access worked fine after enabling universal passwords. SFTP access worked fine as well, as long as the user logging in had sufficient permissions.

OK, there you go. Here’s hoping it may prove useful to someone. Feel free to correct me, clarify these notes, or just tell me I’m crazy in the comments below.

Reader Jeffrey Spear contacted me a while back with some problems he was experiencing in trying to integrate some Linux systems into Active Directory. Basically, Kerberos was working but LDAP wasn’t. He was able to use “kinit <AD username>” to generate a Kerberos ticket, but using the “getent passwd <AD username>” was not working. No error messages, nothing; it just didn’t work.

We traded e-mails back and forth for a while, and eventually he found the solution himself:

We work with a locked down version of OSs and in this case a domain policy on the Windows server was preventing the RHEL machines from accessing account info.  The policy was “Domain controller: LDAP server signing requirements” which was set to “Require signature.”  When I changed this setting to “None” it worked great.

This is good information and important to keep in mind; I’ll be sure to incorporate this into the next revision of the Linux-AD integration instructions. (No, I don’t have a timeframe on when that will be!)

In the meantime, if anyone has a workaround for this problem that will allow LDAP to work with signatures enabled or required, I’d love to hear it. Speak up in the comments below!

A customer with whom I had worked to fully integrate their ESX Server systems into Active Directory—using my instructions here—had run into a problem. The problem was that local logins were refused when the Service Console lost network connectivity. Clearly, this was a problem; if the customer couldn’t login as root when the network is down, even locally at the console, then we have problems. So I set out today to isolate and fix the problem.

After much trial and error, I had determined what was not the cause the problem:

• The /etc/pam.d/system-auth file was not at fault; we tried numerous combinations in system-auth and there was no difference in behavior
• The /etc/ldap.conf file was not at fault; we even tried adding a few additional entries (like “bind_policy soft”) to help with issues when LDAP was down and not responding
• A lack of DNS resolution was not the problem; the behavior was the same whether DNS was working or not

Finally, I was able to track down this thread which discusses the behavior of the nss_ldap libraries when the LDAP service is not available across the network. In this specific message in the thread, it is noted that nss_ldap will try to contact LDAP to enumerate group membership, even if LDAP is down. So the problem was with using LDAP for group membership, and a quick edit to /etc/nsswitch.conf to remove LDAP from the group line proved that to be true.

As shown in the message, the only workarounds are:

• Upgrade to v245 of nss_ldap, which allows the use of the “nss_initgroups_ignoreusers” directive; this instructs nss_ldap to not perform group enumeration for the specified users; or
• Remove the “ldap” entry from the group line in /etc/nsswitch.conf.

Unfortunately, ESX Server 3.0.2 and ESX Server 3.5.0 only supply nss_ldap v207-17, which is too early to support that directive.  Of course, we can’t really upgrade the library, either, since that’s not supported by VMware. So the only real fix for VMware ESX Server AD integration scenarios is to not use Active Directory for group memberships. User accounts can still be managed using Active Directory—and authentication occurs against Active Directory—but groups and group membership will have to be handled locally.

This issue is applicable to other operating systems besides ESX Server, though, and for those operating systems an upgrade of the nss_ldap library and the use of the “nss_initgroups_ignoreuser” directive in ldap.conf may be all that is needed to fix an issue with local logins being refused when network connectivity is not present.

UPDATE: It appears that local logins will work without network connectivity, even with full Active Directory integration, if you use the Emergency Console. Thanks to Magnus for the update!

In my previous article about using NetApp multi-mode VIFs with Cisco switches, I mentioned that you could—at that time—only use 802.3ad static link aggregation:

Be aware that Data ONTAP’s multi-mode VIFs are only compatible with static 802.3ad link aggregation; you can’t use PAgP (Cisco proprietary protocol).  I would assume dynamic LACP is also incompatible.  For this reason we used the “channel-group 1 mode on” statement instead of something like “channel-group 1 mode desirable”.

I recently got some feedback from a NetApp SE in my area; this SE informed me that Link Aggregation Control Protocol (LACP, part of the IEEE 802.3ad specification) is indeed supported with Data ONTAP version 7.2.  This KB article on the NetApp NOW site (login required) indicates that ONTAP 7.2.1 is required in order to use a LACP VIF.

There are a couple important requirements to note; these are laid out in the referenced KB article:

1. Dynamic multimode VIFs should use IP address-based load balancing.  This means that the Cisco switch or the channel group must also use IP address-based load balancing.
2. Dynamic multimode VIFs must be first-level VIFs.  This makes sense; LACP is a Layer 2 protocol, so layering a LACP VIF on top of other VIFs just doesn’t work.

To create the dynamic multimode VIF on the Data ONTAP side, the command is pretty simple:

vif create lacp <vif name> -b ip {interface list}

On the Cisco side, the commands are very similar:

s3(config)#int port-channel1
s3(config-if)#description LACP multimode VIF for netapp1
s3(config-if)#int gi0/23
s3(config-if)#channel-protocol lacp
s3(config-if)#channel-group 1 mode active

These commands would be repeated for all physical ports that should be included in the LACP bundle.  Note the differences from the earlier commands in the previous article; here we use “channel-group 1 mode active” instead of “channel-group 1 mode on”.  We also added the “channel-protocol lacp” command.

Together, these commands will establish a LACP-based link aggregate between a NetApp storage system running Data ONTAP version 7.2.1 or higher and a Cisco IOS-based switch.

Thanks to Jeff, our NetApp SE, for providing the updated information.

Since I had CentOS 5 up and running on ESX Server in the test lab, I decided to try to validate my latest Linux-AD integration instructions on this installation.  Unfortunately, the instructions do not seem to work well at all with CentOS 5; here are some of the errors that I ran into:

• When using “net ads join” to join the Active Directory domain, it didn’t recognize any existing Kerberos tickets.  I’d already run a “kinit <AD username>”, but “net ads” continued to either a) try to use the root account if I didn’t specify the “-U <AD username>” parameter, and b) prompt for password even when I’d already obtained a Kerberos ticket for the specified username.
• When initially trying to join the Active Directory domain, “net ads join” threw this error:
  [2007/12/04 12:57:08, 0] libads/kerberos.c:create_local_private_krb5_conf_
for_domain(594) create_local_private_krb5_conf_for_domain:
failed to create directory /var/cache/samba/smb_krb5.
Error was Permission denied
  This error persisted until I manually created the /var/cache/samba/smb_krb5 directory myself.  Why this directory wasn’t created automatically during the Samba installation is beyond me.  Once I created that directory, the error went away, but Samba still wouldn’t create the keytab or add entries to the keytab.
• The “net ads keytab” command failed miserably; it would not create a keytab, nor would it add entries to a keytab.  No error message is reported; it just doesn’t work.

I inquired on the #samba IRC channel on irc.freenode.net, but the only person willing or able to respond didn’t have any information to provide (in fact, he’d actually used my Solaris-AD integration instructions as a guide for some of his own work).  Various Google searches also failed to provide any helpful information.

By the way, these tests were performed on a stock installation of CentOS 5, with all the latest packages installed using “yum update”.  The Samba version was 3.0.25b-1.el5_1.2.

In the end, I’ve given up on trying to make Samba work in the AD integration process and will instead fallback to the use of ktpass.exe to create the keytab file.  If you have any useful information to share, please let me know or post it in the comments.  Thanks!

This afternoon, I walked back through my own instructions for integrating Solaris 10 and Active Directory, and I found that the process wasn’t as smooth as perhaps I’d believed it to be.  As a result of walking back through the process again myself, I’ve collected some notes.  At some point in the near future, these notes will be integrated into a new version of the Solaris-AD integration instructions.

So, without further ado, here are the notes I collected in no particular order:

• The Blastwave Samba package does not create it’s own smb.conf file in /opt/csw/etc/samba.  This is correctly pointed out in the latest integration instructions, but I wanted to mention it again here.  You’ll need to manually create the /opt/csw/etc/samba/smb.conf file before attempting to join the Solaris server to Active Directory via the ‘net ads join’ command.
• The defaultServerList portion of the ‘ldapclient manual’ command only supports IP addresses.  The LDAP client service kept going into maintenance mode when using hostnames.  On a hunch, I substituted IP addresses for hostnames, and it worked.  Go figure.
• Apparently, you can’t use ‘ldapclient mod’ to change an existing attribute map.  I had a hunch about resolving a co-existence issue where both Solaris and Linux are both authenticating against Active Directory—more on that particular topic is coming soon as well—and needed to change the attribute maps for the homedirectory and loginshell attributes.  I ended up editing the ldap_client_file manually (found in /var/ldap; must be made writable using chmod) in order to make the change.  If anyone has a more elegant fix, please let me know.
• The ‘net ads join’ command correctly creates a Kerberos keytab with the appropriate entries, but places it in the wrong location.  On my test system, it placed the krb5.keytab file in the /etc directory, and Solaris expected it to be in /etc/krb5 instead.  Until I moved that file, authentication against Active Directory consistently failed.
• It turns out that it’s not really necessary to enable the DNS client using ’svcadm enable svc:/network/dns/client:default’; from what I’ve been able to gather, that’s there as a dependency only.  The ‘nslookup’ and ‘host’ commands seemed to work just fine with this service still disabled.

Again, I’ll be incorporating these changes into a future version of the Solaris-AD integration instructions.  I hope to have that complete within the next week or two, so stay tuned.  In addition, I have information coming to help with the co-existence of multiple UNIX and UNIX-like operating systems all authenticating against the same Active Directory forest, so keep your eyes peeled for that as well.

A reader kindly shared with me some tips and tweaks he used to help resolve some performance issues with Linux-AD integration, and now I’d like to share them with you:

I ran into some nagging performance issues with Linux/AD integration the other day, and managed to solve them (mostly)—I thought you might be interested in the solution.

Since I’m not exactly following your integration guide (I use DNS lookups to locate LDAP servers in AD, and use GSSAPI authentication for nss_ldap instead of a binddn), I have a bit more overhead on my getent system calls, that was starting to get noticable when it came to getting directory listings, etc.

Two things I have done to alleviate this issue:

1. Since we have a flat catalog, and all of our DCs are also GCs, I
   disabled recursion. This is not universally relevant, but it does assist me
2. I edited /etc/nscd.conf to set a 5 minute cache time for passwd and group (but not host!) entries. Data is still usually piping fresh, but now we only call nss_ldap once every 5 minutes, instead of every time we need to know who owns a file. Then I started nscd, and set it to start on boot.

NSCD is a standard part of most RHEL and derivative installs.

Since implementing the above, the user-level experience is no longer
discernably different than the old /etc/passwd method of authentication/authorization.

This is good information to have.  Thanks to Brandon for sharing his experience!

I’ve received some feedback from a reader who alerted me to some sort of interaction between the Local Security Policy on the Windows side and Linux servers authenticating to Active Directory via Kerberos/LDAP/Samba.  I haven’t quite been able to get to the root issue yet, but here’s the high level overview.

The reader was seeing strange delays at the end of a Linux logon process that seemingly could not be explained.  After jumping through all the hoops, another administrator within the organization changed the Local Security Policy setting that governed the use of LM and NTLM authentication, and the delays disappeared.

The policy had been set to allow both LM and NTLM authentication; when changed to allow only NTLM authentication, the delays disappeared immediately.  The Linux server in question did have Samba installed, so apparently Samba was timing out trying the LM authentication; this caused the delays.  Of course, this is all just speculation, as we don’t know exactly why the policy change eliminated the delay.

In any case, since I’ve been pushing the use of Samba in my latest integration instructions (Solaris version here), I thought it might be prudent to mention this feedback.  In the event you start seeing some strange delays in your Linux authentication requests, check the Local Security Policy and see if LM authentication is being permitted.  That might just be your culprit.