Really global environment variables for ssh

<mopey> how do I export a variable in pvm?  I add it to my .bashrc or .profile but it ignores it.
<mopey> an environment variable
<mopey> Because I get this error:
<mopey> The value of the $PVM_ROOT environment
<mopey> variable on compute-0-1 is invalid (“”).
<mopey> Use the absolute path to the pvm3/ directory.
<mopey> but if I ssh into compute-0-1, echo $PVM_ROOT it is set correctly
<staynalive_> mopey: I don’t know much about PVM
<staynalive_> but I would check to see if it gets set for non-login sessions
<staynalive_> by doing (in one command) “ssh compute-0-1 printenv”
<staynalive_> Yeah, I just tested it and that’s the issue.
<mopey> good call, it’s not being set for some reason, although it’s “being set” in ~/.bashrc
<mopey> where would I set it, if not bashrc?
<staynalive_> Umm
<mopey> my .profile calls bashrc btw, although that shouldn’t really matter since that’s only on interactive logins, right?
<staynalive_> Yeah
<staynalive_> I think I actully changed a ssh flag to carry the environment variables through to the new machine in a ssh session.
<staynalive_> “PermitUserEnvironment yes”
<mopey> oh.  well that’s handy.
<staynalive_> That way if users set something up funky they can carry it to the nodes.
<staynalive_> But the manual warns of some possible security issues…
<mopey> if someone is on my frontend node, it’s only being used on the compute nodes, so it shouldn’t be that big of a deal.
<mopey> since they are basically thin
<mopey> aaah, ssh has env variables all of it’s own…
<mopey> I remember telling you that at one point.  I guess I’m losing my marbles

<mopey> has anyone ever gotten sshrc to succesfully set ssh environment variables?
<mopey> it *should* be straightforward
<mopey> the sshd man page says:  8.  If $HOME/.ssh/rc exists, runs it; else if /etc/ssh/sshrc exists, runs it; otherwise runs xauth.  The “rc” files are
<mopey>  given the X11 authentication protocol and cookie in standard input.
<mopey> is this not run (when I do ‘ssh compute-0-0 env’) because it is too late in the process?
<mopey> because my $HOME/.ssh/environment _is_ run, and I can set them that way.  Except that I want to do it for all users and that seems to be a lame solution.
<mopey> It seems like the command should be executed *after* the rc files are read (it’s step 9)
<mopey> So I wonder why the hell it’s being ignored…
<mopey> staynalive, you said you use “PermitUserEnvironment yes”.  So do you just set a $HOME/.ssh/environment for each user?

<mopey> haha
<mopey> for those who care, pam overrides my ssh variables for the most part
<mopey> so you can define variables in /etc/security/pam_env.conf
<mopey> god, that took forever to figure out why my /etc/environment variables weren’t getting set over ssh
* Vog-work has quit (“ChatZilla 0.9.79 [Firefox 2.0.0.10/2007111504]”)
<twinprism> thanks for sharing, mopey, I care…
<mopey> weird.
<mopey> :)
<_sera> I don’y
<_sera> sheesh… don’t
<mopey> Normally it probably doesn’t matter I’m sure.  Like if you get a bash shell and can actually execute profile/bashrc
<mopey> But if you have a crippled pvm shell or something, it’s way important
<mopey> plus I think pam_env is how PATHs and junk get set on login – at least on ubuntu
<mopey> *gdm/kdm/xdm login

Build a Bridge and Get Over it

In my environment, the best kind of firewall is the bridge kind of firewall. Meaning: a transparent kind of firewall. Here’s how I generally set them up.

So, I don’t have control of the gateway routers. I have IP addresses all over the place in my little IP pool. For example I have one little server room with a 134.50.7.24 a 134.50.7.88 and a 134.50.7.244. What I want is a firewall complete with IDP to limit access to stuff in a centralized way. BTW, I also tend to have firewalls on every machine, I guess for the double layer of protection. Firewalls are sort of like condoms. Double layering doesn’t seem to help that much, but it couldn’t hurt. And either way, once you’re penetrated, you’re fucked.

To clarify, what I want is something completely invisible to the outside. As far as the outside is concerned, there is nothing there. In fact, this could be the case as far as the inside is concerned. However, you may want to give the bridge an IP address and let something through so you can ssh in to fix whatever problems.

So:

Network —– Bridge ——switch —Internal Network of various IPs

The cool thing about a bridge is you can stick it anywhere and it won’t change how any of your other hosts are hooked up.

For this task, I chose everyone’s favorite os, Linux. I also chose a minimalist version of Ubuntu since I heart ubuntu. This is actually a pretty trivial thing to set up.

# apt-get install bridge-utils

Then you want to create a new bridge device and add your ethx to it. Your ethx will obviously probably be eth0, eth1, eth2 or whatever network devices you want on the bridge.

# brctl addbr br0
# brctl addif br0 eth0
# brctl addif br0 eth1
# ip link set br0 up

Now guess what. You’ve now turned your $300 machine into a $10 hub.

Next if you want to give your bridge an ip address and a route:

# ip addr add x.x.x.x/x brd + dev br0
# route add default gw x.x.x.x dev br0

Where x is your ip and your subnet and gateway.

I find as I’m writing this I’m just repeating what else is out there. Go and read http://www.linuxjournal.com/article/8172 and it will show you the way.

Now that you’ve got your bridge working you can filter traffic using ebtables (on the hardware level) and iptables (on the ip level) and snort and whatever. There are tons of examples of this online, so I won’t bore you here. Awesome.

Put all this in rc.local or whatever boot up script so that your system remembers everything and you’re golden! To all those people who say “just use smoothwall and don’t worry about iptables” in your face! smoothwall/ipcop/monowall are all pretty cool, but being a transparent bridge isn’t one of the things they are capable of out of the box. They are mostly meant to be gateways or whatever.

Matching Regular Expressions that don’t end with…

Regular expressions do not mix well with syntax that requires memory, such as XML. I was trying to add a <br /> tag to every line that did not have a </p> tag. so for example I can print the strings I want with grep -v ‘^.*</P>’

Anyway, this turns out to be a bear, because (?!expression) just isn’t working for me with sed, although I think google says it should.

So what do I do? I make two!

s/(w.*</p>)/1<br />/g

adds a <br /> to every line with a word.

s/(.*</p>)<br />/1/g

take off the <br /> for lines that have a </p>

The good thing about this is it should work with all standard regular expressions, unlike that look ahead stuff which may only work with certain utilities.

You could run this with sed, vim, perl, python, whatever, and it will work.

Hanging mount (not the way you like it)

Symptom: trying

$ mount -t nfs 10.0.0.1:/whatever /whatever

is just hanging there.

Troubleshooting: rpcinfo -p servername is fast and responsive. A lot of times when rpc fails nfs will fall on a backup protocol that is fairly slow. With rpcinfo -p, you can rule this out as being your problem.

portmapper is running. According to lord google, this has been the cause of the problem many a time. Not my problem though.

The server logs don’t say crapola.

What do you do?

Solution: You may just have bad syntax. Try

$ mount -t nfs 10.0.0.1:/whatever/ /whatever/

Yes, this took me hours to figure out. Stupid slash.

From #islug (I kept the last bit because I like to talk about fscking whenever I get a chance).

<mop> Have you ever spent all day stressed out about a problem, then you figure out that it’s all just because you left a trailing front slash?
<mop> Reminds me of my first days programming and missing semi-colons, the bastards.
<twinprism> yes. :)
<TomHung> echo $iknowyourfeeling
<bradmw> mop: what language?
<mop> Today: just mounting an nfs file share (around 2 hours debugging to find out its because I mounted /whatever instead of /whatever/)
<mop> Back in the day: C
<bradmw> does linux have a scandisk type program to check floppies for bad sectors?
<mop> you can fsck a floppy
<mop> just like an apple pie
* TomHung <= fsck up
<TomHung> FSCK U & TH3 HORSE U R0DE IN ON
<bradmw> let me unmount my sheep first
<mop> common mistake. umount.
<mop> ;)
<TomHung> mount /dev/usb /home/sheep

Believe it or not I’ve done basically the same thing with grep -R…

Get Mail List from LDAP

Although bash is my first scripting language, it seems I am becomming more and more of a python convert. Today I needed to send an email to everyone with an account on my ldap server. Normally, I would have used something like sed and bash. Although I did still end up using bash, I forced myself to write the regular expression part in python.

The idea with languages like python is to make it so that it’s easier to write your own bit of code to do what you want than it would be to find someone else’s code and use it. This code is probably pretty useless to most people. It is not even a little efficient. But who knows, I may help a budding sysadmin who is just starting to write his own scripts. And by blogging it, hopefully I know where to find it for next time.

Here is the script to pull out user’s email addresses, one per line (which is exactly the type of file my mail client will accept to make an email list):

"""
mailfind.py

This program was written to get all the email addresses from raw input
and print them onto the screen
"""
import re
    while 1:
        try:
            line = raw_input() + 'n'
            string = re.search(r"(w*.)?w*@w*.w*",line)
            try:
                print string.group()
            except:
                continue

        #this should only happen on the last iteration
        except:
            break

I ended up just using some trivial bash stuff to do some of the processing, although this would be pretty easy to have built into the python, but it was even easier to just put it in good old familiar and unmanagable bash.

#!/bin/bash
slapcat | ./mailfind.py | sort | uniq

Awesome. Now I have my mail addresses to use for my black-market viagra selling business.

Common Permission Error

One extremely important part of using any multi-user Operating System is correctly understanding permissions. Over the past couple years, I have been administering a Linux server with a lot of users, and there is one error that has popped up over and over again.  A lot of users forget or don’t realize that permission to delete a file from a directory is determined by the write flag of the directory, not the write flag of the file.

Take this scenario: Bob is an unprivileged user on a LAMP box and he is hosting a dynamic website.  He wants to allow changes to folders within his public_html directory so he can do things like upload templates, media, etc.  However, the apache user is not Bob, it is something like www-data.  As an unprivileged user, Bob cannot chown or chgrp a directory to a group he doesn’t belong to.  He might think about contacting the sysadmin, but more likely he chmods the directories to be a+w, and is careful to make the files not have the write flag (assuming that this is what determines if a file can be deleted or not). This is an incorrect assumption, and he is leaving his files to be deleted by whoever else has an account on the server.

For example:

bob@lamp:~$ ls -l     #note the test directory has o+w
total 4
drwxrwxrwx 2 bob stupid 4096 2008-07-01 10:27 test

bob@lamp:~$ cd test/

bob@lamp:~/test$ ls -l myfile     #note that myfile does not have o+w
-rw-r–r– 1 lundeen2 stupid 5 2008-07-01 10:28 myfile

bob@lamp:~/test$ su otheruser

otheruser@lamp:~/test$ rm myfile     #other random users are able to delete this file
rm: remove write-protected regular file `myfile’? y

otheruser@lamp:~/test$ ls -l
total 0

There are several ways to handle Bob’s situation.  Bob could ask a privileged user to add him to the www-data group (though this won’t work very well if all the users are part of this group) or he could ask the admin to setattr +i the file to make it undeletable (though he himself could not delete it afterward). A better way would probably be for Bob to use acls (eg setfacl) or to set the sticky bit on the directory (chmod +t).  The sticky bit might be good enough, as it is probably what Bob wanted in the first place – for other users besides himself to be able to write to his directory, but not giving those users a chance to delete his or www-data’s files. Using acls is probably best, but can also be slightly more complicated.

While the behavior of permissions may be obvious to a system administrator, to an average user, it seems it is not.  In my weekly cron scripts I have a  “find / -type d  ( -perm -o+w  -perm 1000 )” to search for all files with this permission.