Tagged: unix

Zsh glob qualifiers ftw!

I have a FreeBSD 7.1 server, a couple of macs, a windows machine, and an ubuntu machine. I need to share files between them all. I could try to get NFS for windows working, but it seemed to me that using Samba was a good way for me to allow all of the machines (Even the windows one!) have access to the storage I have on my “server”. I’ve been working with BSD, Linux and Windows for a long time, and I still get emails to thank me for that old Linux From Scratch hint I wrote that describes how to set up printing with samba and cups. I thought this would be a cinch. Was it? Of course not haha. It turns out that even though my user had access to mount the share, I had mounted it in FreeBSD owned by user root, group wheel with permission set to 755. My user is in the wheel group and I would like the wheel group to have write permission, so I remounted it with the following line in /etc/fstab:

/dev/ad8s1  /mnt/sambashare msdosfs rw,large,-m=775,-g=wheel 0 0

the large is there because it’s a 500GB drive.

And now, all is well. I can mount the share and I have read write access! I can create and delete a test folder… so that’s it! Right? Wrong. There’s an old windows directory on there that I need to delete, but OS X says I haven’t got enough permission to do that. Oh no!

Well how am I supposed to go digging through some directory tree and find the files that I don’t have permission to delete? I bet there’s a unix command that can help me… (actually I know there is – it’s called find, and find is great…but… I love zsh and I love finding fun reasons to use its features!)  Since I know ls will list my files… and ls -al will tell me the permissions… why not just say “ls -al everything i dont have write permission to”?

ls -al *(^I)

The stuff that is between the parentheses is called a glob qualifier. Glob qualifiers let you ask zsh to give you back more specific information. In this case, the capital I means group writable files and the ^ is used to negate the qualifier. If you try to translate this command line into english it would say something like, “list the names and permissions of all of the files in the current directory, even the hidden ones, that are not group writable.”

You may be thinking, “Wait… did you say all files in the current directory?” and if you are thinking that – you’re right. The above command only lists the files in the current directory, so it is not very useful right now.  I need to look in my mounted drive, which is on /mnt/sambashare, so how about this command line?

ls -al /mnt/sambashare/*(^I)

That is close… but not quite there. It’s only showing files in that directory, and I need to look into all the directories that might be in there. Since all the files and directories in that directory are group writable, this gives me:

zsh: no match

So how can I tell ls to look into directories too? That’s easy! I can use the recursive globbing operator, **

Now the command line looks like this:

ls -al /mnt/sambashare/**(^I)

But that STILL doesn’t go all the way into directories I need it to. This is getting crazy now isn’t it? It turns out that to look into directories, even directories that are inside other directories, we cannot simply use the recursive globbing operator, we need to append a /* as well, which makes the command line:

ls -al /mnt/sambashare/**/*(^I)

But now… oh no! We’re still not there! This returned a list including some files that ARE group writable! Why don’t I just run chmod -R  and end this agony already? (Is that what you’re thinking? haha well the answer is… I want to know exactly what files I can’t delete before I take further action). It seems that the ls command is going into directories and showing me things I didn’t ask for. I have a hunch that it’s because ls is doing some recursion of its own… (don’t ask me why, it’s just a hunch) so I’m going to add the -d option to the ls command, making the command line….(drum roll please):

ls -ald /mnt/sambashare/**/*(^I)

….and my hunch was correct! So now I have my list of all files in my share that are not group writable. It might seem like a lot of time and effort, but for a seasoned zsh user this is nothing! I did have to consult the zsh man page to find the glob qualifier for “group writable files”, but that didn’t take long. In the next article I’ll continue with this scenario, and tell you if I ever did manage to get those files deleted 🙂 Stay tuned… if you dare!

Wicked Cool Ruby Scripts

Having fun and solving problems can be mutually exclusive. Even for professional programmers and system administrators who chose their career because they enjoy problem solving, there can be times when finding a solution is an exercise in the mundane. Luckily, there are tools designed to ease the pain and frustration faced by programmers and admins. Ruby is a programming language that was designed from the start to not only provide a means of solving problems, but also to be inherently intuitive and fun to use. Wicked Cool Ruby Scripts, by Steve Pugh, is a book aimed to bring to light the fact that you can use Ruby to write concise yet useful scripts that solve difficult problems.

If you’re a fan of the “Wicked Cool” books from No Starch Press, you’ll find the format of this book familiar. It’s not a hefty tome complete with syntax and “hello world” introductory lessons, rather it’s almost a recipe book of sorts, divided into sections of problems and chock full of immediately useful Ruby code. This is the “Wicked Cool” book I’ve been waiting for, because although I write PHP and shell scripts (not so much Java and Perl, other topics covered in the series), I’ve always thought Ruby was the coolest of all. Right from the start, you can tell that Steve Pugh agrees with me. His tone throughout the book is that of a friend who has something fun to share, never browbeating or lecturing. He’s not simply writing to show us that he knows how to write Ruby well, he’s really trying to help us out.

Honestly, some of the examples in Wicked Cool Ruby Scripts might leave you wondering why you’d use such a powerful language like Ruby for such seemingly simple things. What Steve Pugh tries (and succeeds!) to show us is that Ruby isn’t just for writing massive web applications, but it can also handle tasks often relegated to the ubiquitous, but cryptic Awk or shell languages. Perhaps you still wonder why you’d want to? “Just because you can doesn’t mean you should”, right? So why bother? Because Ruby is “Wicked Cool”, that’s why.

So what’s cool? How about a simple file alteration monitor to help you see what’s changed on your system? Not cool enough? How about a web based photo gallery in about 50 lines of code? Still not impressed? How about writing a Metasploit module to attack one Windows machine from another? From general purpose utilities to system security and yes, even some games, Wicked Cool Ruby Scripts has enough in it to pique the interest of just about any programmer or sysadmin. I for one am finding it hard to concentrate on this review because I want to get back to writing Ruby. If you’re a programmer waiting for a good excuse to try Ruby, or a Windows sysadmin wondering what an open source programming language can do for you, you’ll find Wicked Cool Ruby Scripts enlightening, inspiring, and of course… cool.

PHP/Recode fail

I use PHP on FreeBSD servers, compiled from the ports system. It works very well, except that sometimes… well… sometimes… segfaults happen. Yes random, bizarre, “how did this working system suddenly become non-working wtf” errors happen.

Here’s an example of what happened to me today. This is a snip from my lighttpd error log

(mod_fastcgi.c.2722) child signaled: 11
(mod_fastcgi.c.1051) the fastcgi-backend /usr/local/bin/php-cgi failed to start:
(mod_fastcgi.c.1065) terminated by signal: 11
(mod_fastcgi.c.1070) to be exact: it segfaulted, crashed, died, … you get the idea.
(mod_fastcgi.c.1072) If this is PHP, try removing the bytecode caches for now and try again.
(mod_fastcgi.c.2759) ERROR: spawning fcgi failed.


Well in a nutshell, php-cgi is failing to start. When I attempt to run /usr/local/bin/php-cgi from the command line, it causes a segmentation fault. If you’ve seen this before, you might have thought “oh no… sig11! signal 11 could mean my hardware is fried!” At least I thought so. The truth of the matter, in this case, is that the php-cgi executable is not happy. Why isn’t it happy? I’m glad you asked. We all want our executeables to be happy, right?

This wasn’t easy for me to figure out the first time it happened to me last year. I was going nuts trying to figure out what went wrong, I even put in a request for new hardware (yes a new development server, which I actually got). This didn’t solve my problem either. I began to panic. I’m not sure what made me think of it, but after a while I decided to try minimalizing my PHP installation, by commenting out extensions from my extensions.ini file (on my sys it’s /usr/local/etc/php/extensions.ini)

With everything commented out, php-cgi was happy again, but of course I need at least a few of the extensions (specifically mysqli), and so I started adding extensions back in one at a time to find the problem. It turns out that the problem is with the recode extension.

With the recode extension out of the picture, things went along fine for quite a while and I was able to develop my small web application with Zend Framework (1.5 at the time, I think). PHP was happy, Lighttpd was happy, I was happy. There was a lot of happiness to be found! And then it happened. I updated my PHP port with portmaster. A dark cloud (a cloud named sig11) loomed over the server and the happiness turned to confusion once again. Then I remembered… “didn’t this happen last year?” Yes. Yes it did. Same problem. This time I didn’t panic though, I went straight for the extensions file.

Now I’m not sure what about the recode extension my system doesn’t like, but it doesn’t like it. It didn’t like it last year, and it doesn’t like it now. The PHP port installed a new copy of extensions.ini which includes the recode extension and so that’s why the error reoccured after updating the port. If you are experiencing weird segmentation faults with PHP, I highly recommend you start commenting out your extensions. Also, you may want to check the order of the extensions. If they are in alphabetical order, something is wrong. Some extensions rely on others to be loaded first, so be careful.

That’s it for now… PHP is happy once again, and I’m working on a new dev project, this time with Zend Framework 1.7  🙂 I’d tell you more about it… but it’s a secret. No, really, it’s a secret.

ISC DHCPD duplicate uid lease

Ever see this?

dhcpd: uid lease for client xx:xx:xx:xx:xx:xx is duplicate on 192.168.1/24

or something like it, in your dhcp logs? Well I checked /var/log/messages today and saw that I had thousands of this message repeated over and over (so much so it was spamming my log  and making it harder to find what might be important stuff). I searched around the ‘net and found some hints but nothing conclusive. The hints pointed toward a duplicate lease (duh) but more specifically lead me to check my leases file. Mine was located in /var/db/dhcpd/dhcpd.leases.

It turns out that what caused this error for me was that I had started the machine in question (the MAC address has been altered to protect the innocent haha) before I had the time to create a reservation for it. Then after it already had an address leased to it, I created a reservation for it (for internal DNS purposes, nothing more). Apparently because duplicates are allowed by default in isc dhcpd, when the machine’s networking was restarted and it got it’s new reserved IP, the server kept the original lease (which was that .150 address) and also got a lease from it’s reservation (.80 on my network). Since the .150 lease was still present in the leases file it caused the warning message (over and over and over for a month).

So if you are using isc dhcpd, and you use reservations, and you’re getting these error messages, you might want to check your leases file and make sure that you don’t have a lease for something that also has a reservation under a different IP.

The other pages I found on my search also suggested that you might have a reservation that hands out an address that is within your dynamic scope. You definitely don’t want to do that, so you may need to check that out as well in your dhcpd.conf.

For the curious, I run FreeBSD 7 on a Pentium 4 Dell desktop (I think it’s a dimension series, I haven’t looked at it in a while haha) that sits in my kitchen as a headless DNS/DHCP/Web/Random Unix Fun server. It’s actually making lots of bad noise lately and I think one of it’s fans might be going 🙁 I’ll check that out and report back 🙂

Sed on Mac OS X 10.5 Leopard

Imagine you’re a PHP developer that uses OS X. You’re given 50+ php files that all have a line that needs to be changed. 50 files, one change? Hmmm sounds like maybe I could use automator to do this! Well I don’t know how to use automator 😉 Luckily, I’m a unix geek, and even more luckily, OS X has a fairly strong unix back end and a great terminal emulator. So how does that help? Well if you’re a unix geek you know that this sort of problem just screams “SED! use SED! this is what SED IS FOR!” And it’s true. This is where sed is great.

Say you want to change this line:

 $config_file = $_SERVER['DOCUMENT_ROOT']."/dev/config.php";

To this:

 $config_file = "dev/config.php";

As a matter of fact… wait a minute! I don’t want to just change that line… no I want to be able to show what I’ve changed, so that the next person who looks at this can see what it used to say. In this instance it’s also important to show the work I’ve done because I’m making changes to someone else’s work. So what I want to do is comment out the line and then add a new line with my change underneath. Sounds a bit more complicated right? Well it’s not really – unless you’re using OS X (you’ll see why in a minute *sigh*). The goal is to end up with this:

 //$config_file = $_SERVER['DOCUMENT_ROOT']."/dev/config.php";
 $config_file = "dev/config.php";

On the linux machine I keep around the office the command to perform this change on the all php files in the current directory looks like this ( I had to split the line to fit on the site):

sed -i .bak "s/^\$config_file = \
\$config_file = \$_SERVER\['DOCUMENT_ROOT'\]\.\"\/dev\/config\.php\"\;\
\n\$config_file = \"dev\/config\.php\"\;/g" *.php

The breakdown of the above command is as follows:

sed | the sed command

-i .bak | the -i option means do this change “in place” and copy the original file to originalname.bak

| we use the double quote here to start the sed command because there are single quotes in the command

s/^\$config_file =
= \$_SERVER\[‘DOCUMENT_ROOT’\]\.\”\/dev\/config\.php\”\;\n\$config_file = \”dev\/config\.php\”\;/g
| this is the magic of sed. it looks for the line we want to change, adds // to the beginning of it, adds a new line after it, and then adds the text that we want after the new line. Amazing isn’t it?

| the command ends with the double quote

and, finally:

*.php | represents all of the files that end in .php in the current directory.

Now if you thought that was a mess, check out how that command needs to look in order to work on OS X (10.5.1, possibly other versions)

sed -i .bak "s/^\$config_file = \
\$config_file = \$_SERVER\['DOCUMENT_ROOT'\]\.\"\/dev\/config\.php\"\;\
\$config_file = \"dev\/config\.php\"\;/g" index.php

The crazy part here is the need to use \\”$’\n'”\

What that represents is an escaped version of the literal newline character. With the version of sed currently in OS X, that is the only way (that I could find) to add a newline character with sed. Now you know too.

So that concludes this edition of mac unix geekery… until next time…