Kerberos

You are currently browsing articles tagged Kerberos.

I had a reader contact me with a question on using Kerberos and LDAP for authentication into Active Directory, based on Active Directory integration work I did many years ago. I was unable to help him, but he did find the solution to the problem, and I wanted to share it here in case it might help others.

The issue was that he was experiencing a problem using native Kerberos authentication against Active Directory with SSH. Specifically, when he tried open an SSH session to another system from a user account that had a Kerberos Ticket Granting Ticket (TGT), the remote system dropped the connection with a “connection closed” error message. (The expected behavior should have been to authenticate the user automatically using the TGT.) However, when he stopped the SSH daemon and then ran it manually as root, the Kerberos authentication worked.

It’s been a number of years since I dealt with this sort of integration, so I wasn’t really sure where to start, to be honest, and I relayed this to the reader.

Fortunately, the reader contacted me a few days later with the solution. As it turns out, the problem was with SELinux. Apparently, by copying the keytab file from a Windows KDC (an Active Directory domain controller), the keytab is considered “foreign” because it doesn’t have the right security context. The fix, as my reader discovered, is to use the restorecon command to reset the security context on the Kerberos files, like this (the last command may not be necessary):

restorecon /etc/krb5.conf
restorecon /etc/krb5.keytab
restorecon /root/.k5login

Once the security context had been reset, the Kerberos authentication via SSH worked as expected. Thanks Tomas!

Tags: , , , ,

A reader contacted me a short while ago to inquire about a problem he was having with his Linux-AD integration efforts. It seems he had recently added a new domain controller (DC) that was intended to be a DC for a disaster recovery (DR) site. When he took this new DR DC offline in order to physically move it to the DR site, some of his AD-integrated Linux systems started failing to authenticate. More specifically, Kerberos continued to work, but LDAP lookups failed. When the reader would bring the DR DC back online, those systems started working again.

There was a clear correlation between the DR DC and the AD-integrated Linux systems, even though the /etc/ldap.conf file specifically pointed to another DC by IP address. There was no reference whatsoever, by IP address or host name, to the DR DC. Yet, every time the DR DC was taken offline, the behavior returned on a subset of Linux hosts. The only difference we could find between the affected and unaffected hosts was that the affected hosts were not on the same VLAN as the production domain controllers.

I theorized that Windows’ netmask ordering feature, which prioritizes the return of DNS lookups to provide clients with addresses that are “closer” to them, was playing a role here. However, the /etc/ldap.conf was using IP addresses, not the domain name or even the fully qualified domain name of a DC. It couldn’t be DNS, at least not as far as I could tell.

Upon further investigation, the reader discovered that the affected Linux servers—those that were on a different VLAN than both the production DCs as well as the DR DC—were maintaining persistent connections to the DR DC. (He found this via netstat.) When the DR DC went offline, the affected Linux hosts tried to continue to communicate to that DC and that DC only. Once the reader was able to get the affected Linux hosts to drop that persistent connection, he was able to take the DR DC offline and the Linux hosts worked as expected.

So, the real question now becomes: how (or why) did the Linux servers connect to the DR DC instead of the production DC for which they were configured? I think that Active Directory issued an LDAP referral to direct the affected Linux servers to the DR DC as a result of site topology. Perhaps due to an incorrect or incomplete site topology configuration, Active Directory believed the DR DC should handle the VLANs where the affected Linux servers resided. If that is indeed the case, the fix would be to make sure that your AD site topology is correct and that subnets are appropriately associated with sites. Of course, this is just a theory.

Has anyone else seen an issue similar to this? What fix were you able to implement in order to correct it?

Tags: , , , , , ,

I’m sorry, folks, but I’m not going to have the time or the resources to publish an update to my existing instructions for integrating Solaris 10 into Active Directory. Quite some time ago I had posted that I planned on creating an update to the original instructions so as to incorporate some lessons learned, but it keeps get pushed aside for other tasks that are more important and more relevant to my day-to-day work. Rather than keep readers hanging on for something that will likely never appear, I’d rather just be upfront and frank about the situation. As much as I’d love to spend some time working on the Solaris-AD integration situation and documenting my findings, I just don’t have the time. Sorry.

Tags: , , , , , ,

I ran across this handy white paper about OpenSSH on Linux using Kerberos authentication with Windows and Active Directory. There’s not a whole lot in there that isn’t also covered in my Active Directory integration notes, but it is useful information nevertheless.

Tags: , , , ,

Reader Jeffrey Spear contacted me a while back with some problems he was experiencing in trying to integrate some Linux systems into Active Directory. Basically, Kerberos was working but LDAP wasn’t. He was able to use “kinit <AD username>” to generate a Kerberos ticket, but using the “getent passwd <AD username>” was not working. No error messages, nothing; it just didn’t work.

We traded e-mails back and forth for a while, and eventually he found the solution himself:

We work with a locked down version of OSs and in this case a domain policy on the Windows server was preventing the RHEL machines from accessing account info.  The policy was “Domain controller: LDAP server signing requirements” which was set to “Require signature.”  When I changed this setting to “None” it worked great.

This is good information and important to keep in mind; I’ll be sure to incorporate this into the next revision of the Linux-AD integration instructions. (No, I don’t have a timeframe on when that will be!)

In the meantime, if anyone has a workaround for this problem that will allow LDAP to work with signatures enabled or required, I’d love to hear it. Speak up in the comments below!

Tags: , , , , ,

A customer with whom I had worked to fully integrate their ESX Server systems into Active Directory—using my instructions here—had run into a problem. The problem was that local logins were refused when the Service Console lost network connectivity. Clearly, this was a problem; if the customer couldn’t login as root when the network is down, even locally at the console, then we have problems. So I set out today to isolate and fix the problem.

After much trial and error, I had determined what was not the cause the problem:

  • The /etc/pam.d/system-auth file was not at fault; we tried numerous combinations in system-auth and there was no difference in behavior
  • The /etc/ldap.conf file was not at fault; we even tried adding a few additional entries (like “bind_policy soft”) to help with issues when LDAP was down and not responding
  • A lack of DNS resolution was not the problem; the behavior was the same whether DNS was working or not

Finally, I was able to track down this thread which discusses the behavior of the nss_ldap libraries when the LDAP service is not available across the network. In this specific message in the thread, it is noted that nss_ldap will try to contact LDAP to enumerate group membership, even if LDAP is down. So the problem was with using LDAP for group membership, and a quick edit to /etc/nsswitch.conf to remove LDAP from the group line proved that to be true.

As shown in the message, the only workarounds are:

  • Upgrade to v245 of nss_ldap, which allows the use of the “nss_initgroups_ignoreusers” directive; this instructs nss_ldap to not perform group enumeration for the specified users; or
  • Remove the “ldap” entry from the group line in /etc/nsswitch.conf.

Unfortunately, ESX Server 3.0.2 and ESX Server 3.5.0 only supply nss_ldap v207-17, which is too early to support that directive. Of course, we can’t really upgrade the library, either, since that’s not supported by VMware. So the only real fix for VMware ESX Server AD integration scenarios is to not use Active Directory for group memberships. User accounts can still be managed using Active Directory—and authentication occurs against Active Directory—but groups and group membership will have to be handled locally.

This issue is applicable to other operating systems besides ESX Server, though, and for those operating systems an upgrade of the nss_ldap library and the use of the “nss_initgroups_ignoreuser” directive in ldap.conf may be all that is needed to fix an issue with local logins being refused when network connectivity is not present.

UPDATE: It appears that local logins will work without network connectivity, even with full Active Directory integration, if you use the Emergency Console. Thanks to Magnus for the update!

Tags: , , , , , ,

Since I had CentOS 5 up and running on ESX Server in the test lab, I decided to try to validate my latest Linux-AD integration instructions on this installation.  Unfortunately, the instructions do not seem to work well at all with CentOS 5; here are some of the errors that I ran into:

  • When using “net ads join” to join the Active Directory domain, it didn’t recognize any existing Kerberos tickets.  I’d already run a “kinit <AD username>”, but “net ads” continued to either a) try to use the root account if I didn’t specify the “-U <AD username>” parameter, and b) prompt for password even when I’d already obtained a Kerberos ticket for the specified username.
  • When initially trying to join the Active Directory domain, “net ads join” threw this error:
    [2007/12/04 12:57:08, 0] libads/kerberos.c:create_local_private_krb5_conf_
    for_domain(594) create_local_private_krb5_conf_for_domain:
    failed to create directory /var/cache/samba/smb_krb5.
    Error was Permission denied

    This error persisted until I manually created the /var/cache/samba/smb_krb5 directory myself.  Why this directory wasn’t created automatically during the Samba installation is beyond me.  Once I created that directory, the error went away, but Samba still wouldn’t create the keytab or add entries to the keytab.
  • The “net ads keytab” command failed miserably; it would not create a keytab, nor would it add entries to a keytab.  No error message is reported; it just doesn’t work.

I inquired on the #samba IRC channel on irc.freenode.net, but the only person willing or able to respond didn’t have any information to provide (in fact, he’d actually used my Solaris-AD integration instructions as a guide for some of his own work).  Various Google searches also failed to provide any helpful information.

By the way, these tests were performed on a stock installation of CentOS 5, with all the latest packages installed using “yum update”.  The Samba version was 3.0.25b-1.el5_1.2.

In the end, I’ve given up on trying to make Samba work in the AD integration process and will instead fallback to the use of ktpass.exe to create the keytab file.  If you have any useful information to share, please let me know or post it in the comments.  Thanks!

Tags: , , ,

This afternoon, I walked back through my own instructions for integrating Solaris 10 and Active Directory, and I found that the process wasn’t as smooth as perhaps I’d believed it to be.  As a result of walking back through the process again myself, I’ve collected some notes.  At some point in the near future, these notes will be integrated into a new version of the Solaris-AD integration instructions.

So, without further ado, here are the notes I collected in no particular order:

  • The Blastwave Samba package does not create it’s own smb.conf file in /opt/csw/etc/samba.  This is correctly pointed out in the latest integration instructions, but I wanted to mention it again here.  You’ll need to manually create the /opt/csw/etc/samba/smb.conf file before attempting to join the Solaris server to Active Directory via the ‘net ads join’ command.
  • The defaultServerList portion of the ‘ldapclient manual’ command only supports IP addresses.  The LDAP client service kept going into maintenance mode when using hostnames.  On a hunch, I substituted IP addresses for hostnames, and it worked.  Go figure.
  • Apparently, you can’t use ‘ldapclient mod’ to change an existing attribute map.  I had a hunch about resolving a co-existence issue where both Solaris and Linux are both authenticating against Active Directory—more on that particular topic is coming soon as well—and needed to change the attribute maps for the homedirectory and loginshell attributes.  I ended up editing the ldap_client_file manually (found in /var/ldap; must be made writable using chmod) in order to make the change.  If anyone has a more elegant fix, please let me know.
  • The ‘net ads join’ command correctly creates a Kerberos keytab with the appropriate entries, but places it in the wrong location.  On my test system, it placed the krb5.keytab file in the /etc directory, and Solaris expected it to be in /etc/krb5 instead.  Until I moved that file, authentication against Active Directory consistently failed.
  • It turns out that it’s not really necessary to enable the DNS client using ‘svcadm enable svc:/network/dns/client:default’; from what I’ve been able to gather, that’s there as a dependency only.  The ‘nslookup’ and ‘host’ commands seemed to work just fine with this service still disabled.

Again, I’ll be incorporating these changes into a future version of the Solaris-AD integration instructions.  I hope to have that complete within the next week or two, so stay tuned.  In addition, I have information coming to help with the co-existence of multiple UNIX and UNIX-like operating systems all authenticating against the same Active Directory forest, so keep your eyes peeled for that as well.

Tags: , , , , , ,

I’ve received some feedback from a reader who alerted me to some sort of interaction between the Local Security Policy on the Windows side and Linux servers authenticating to Active Directory via Kerberos/LDAP/Samba.  I haven’t quite been able to get to the root issue yet, but here’s the high level overview.

The reader was seeing strange delays at the end of a Linux logon process that seemingly could not be explained.  After jumping through all the hoops, another administrator within the organization changed the Local Security Policy setting that governed the use of LM and NTLM authentication, and the delays disappeared.

The policy had been set to allow both LM and NTLM authentication; when changed to allow only NTLM authentication, the delays disappeared immediately.  The Linux server in question did have Samba installed, so apparently Samba was timing out trying the LM authentication; this caused the delays.  Of course, this is all just speculation, as we don’t know exactly why the policy change eliminated the delay.

In any case, since I’ve been pushing the use of Samba in my latest integration instructions (Solaris version here), I thought it might be prudent to mention this feedback.  In the event you start seeing some strange delays in your Linux authentication requests, check the Local Security Policy and see if LM authentication is being permitted.  That might just be your culprit.

Tags: , , , , ,

I was going through my list of flagged headlines in NetNewsWire and realized that I’d built up quite a list of articles that I intended to write something about.  Some of them just don’t merit a full-blown post, though, so I thought I’d just toss a bunch of them in here along with a brief sentence or two about them:

  • VMTN Discussion Forums: vdiskmanager GUI for OSX:  An enterprising Fusion user has written an OS X GUI for vdiskmanager, so that VMDKs on Fusion can be expanded or defragmented, or new virtual disks can be created.  I haven’t tried it yet, but it looks like it could be extremely useful, and it’s nice to see Fusion users creating useful utilities like this.
  • Running ESX 3i Beta in a VM with VMware Fusion:  Still thinking Fusion, this article discusses how a user managed to get ESX Server 3i (the beta version obtained at VMworld 2007) running as a VM under Fusion.  There’s also information on running it under Workstation 6 as well.
  • Tech: How to get the command line in ESX Server 3i beta:  Turns out ESX Server 3i has a command line after all, based on BusyBox.  Richard Garsthagen has more information about ESX 3i available at run-virtual.com.  Also see Eric Sloof’s info on boot options.
  • Storm Worm Botnet Attacks Anti-Spam Firms:  Is this botnet really as massive as everyone says?  I’ve been seeing so many articles about the Storm botnet, but I have yet to see (perhaps I haven’t looked hard enough yet) definitive information that describes the type of traffic these bots generate.  Surely there’s got to be something we can do about this.
  • Microsoft Updates Windows Without User Permission, Apologizes:  Oh, goodness—where do I start with this one?  Let’s just say that I’m glad I’m using Little Snitch, which catches this kind of outbound traffic that so easily slips through the Windows “firewalls” onto the Internet. Otherwise, I might be getting product updates without anyone bothering to tell me so.  (And perhaps it’s just me, but an apology from Microsoft doesn’t make me feel any more trusting of them.)
  • NFS vs iSCSI vs FC:  More information on why we should be interested in running VMware over NFS.

I guess that’s all for now, as it’s getting late and I have to get up in the morning and go to church.  Feel free to share any comments or corrections below.  Thanks for reading!

Tags: , , , , , , , ,

« Older entries