Joseph Scott’s Blog

FreeBSD 7.0 is finally released. For an over view of what’s new check out the release announcement. This was a long time coming, I look forward to giving it a spin.

Last month Kris Kennaway spoke about FreeBSD 7 (pdf). Lots of details about how FreeBSD 7 came about and what you can expect to see when 7.0 finally gets released.

Perhaps the most interesting part though was the SMP focused case study involving MySQL and PostgreSQL on pages 10 through 19. The numbers for 7.0 compared to previous versions of FreeBSD on transactions per second on multi-CPU systems is really impressive. The ULE scheduler gives a huge boost over the traditional 4BSD scheduler. Looks like ULE will become the default scheduler sometime during the 7.x life time (7.1 perhaps?). You want to see CPUs really put to use? Check out page 13.

I was rather surprised to see the MySQL numbers come in so much lower than the PostgreSQL numbers. There seems to be a MySQL specific problem that limits how well it can scale across multiple CPUs.

Earlier this week I came across Robert Watson’s page on Porting WWIV 4.23 to FreeBSD. I immediately had flashbacks of my BBS days from years ago.

It’s hard to imagine that in the 80s and early 90s just about the only way I had access to email and message boards was through BBSs. Made some good friends from all that time spent on BBSs (hi Jay!). Staying up all night writing WWIV mods in C using Turbo C (I had the C/C++ version) from Borland. I recall Turbo C fitting on three 3 1/2 inch floppy disks. That included their editor application as well.

In some ways blogs and forums have become the web versions of what BBSs where back then. Now I just need to write a plugin for WordPress to support doors, then I could run Trade Wars on my blog!

• Box.net Technical Blog | Blog Archive | How to Debug PHP with Vim and XDebug on Linux - Debugging PHP from Vim. Nice.
  Tags: php vim debug xdebug

• Porting WWIV 4.23 to FreeBSD UNIX - Now that is old school hacking!
  Tags: wwiv bbs freebsd

• JavaScript-based Amazon Web Services Simple Monthly Calculator - Quick way to determine how much it will cost you to use Amazon services (S3, EC2, etc).
  Tags: amazon aws calculator ec2 s3

The FreeBSD Foundation has arranged to auction off the first copy of Absolute FreeBSD, 2nd Edition by Michael Lucas. The proceeds of the auction will go to the foundation:

http://cgi.ebay.com/ws/eBayISAPI.dll?ViewItem&item=120175384688&ssPageName=ADME:L:LCA:US:1123.

I didn’t see this mentioned on the foundation web site yet, so here is a link to the email announcement of the auction.

BSDTalk posted an interview with Greg Lehey (a.k.a. Grog) this morning. Greg has a long history in the unix world and has contributed a fair bit to FreeBSD over the years.

I knew that he had worked for Linuxcare during the dot.com boom, but I didn’t realize that he now works for MySQL (and has for almost a year).

Greg is a very intelligent guy, even if he does use emacs :-)

Christian Zander sent an email to the FreeBSD-Hackers list detailing NVIDIA FreeBSD kernel feature requests. Having high quality video drivers is a very good thing:

NVIDIA has been looking at ways to improve its graphics driver for the
FreeBSD i386 platform, as well as investigating the possibility of adding
support for the FreeBSD amd64 platform, and identified a number of
obstacles. Some progress has been made to resolve them, and NVIDIA would
like to summarize the current status. We would also like to thank John
Baldwin and Doug Rabson for their valuable help.

This summary makes an attempt to describe the kernel interfaces needed by
the NVIDIA FreeBSD i386 graphics driver to achieve feature parity with
the Linux/Solaris graphics drivers, and/or required to make support for
the FreeBSD amd64 platform feasible. It also describes some of the
technical difficulties encountered by NVIDIA during the FreeBSD i386
graphics driver’s development, how these problems have been worked around
and what could be done to solve them better.

UPDATE Thu 13 Jul 2006 @ 2:30pm : BSDTalk has an interview with Andy Ritger and Christian Zander from nVidia.

Jeff Bonwick pointed out that ZFS is going to be ported to FUSE (filesystem in user space). This is being done as part of the Google Summer of Code by Ricardo Manuel da Silva Correia.

For more info check out the Google SoC application and the ZFS on FUSE/Linux Blog. Although this project specifically mentions FUSE on Linux I hope that it will work with FUSE for FreeBSD.

I don’t know what sort of performance penalty is involved with FUSE, but it would be darn cool to have this work with FreeBSD.

Colin Percival posted a message to the FreeBSD-stable email list looking for FreeBSD sysadmins to take a security survey. So if you are responsible for keeping a FreeBSD running take a moment to take the survey.

On Monday FreeBSD released 6.1. For the details on what is new since 6.0 check out the release notes for your platform. Now that 6.1 is out I look forward to pfSense hitting their 1.0 release. I’m sure that I could come up with an excuse to try out pfSense once they are out of beta :-)

When I had first started using FreeBSD one of the first things I would do on a new system is install top from the ports collection. It is hard to imagine this now, but back then it wasn’t part of the base system.

But all of that changed in March 1997, nine years ago this month, when top was imported into the base system. It seems like a small thing, but I was thrilled when that happened. One less thing to add on after an install.

For better or for worse I think it takes more than that to impress me now.

There are a lot of open source licenses out there. For me open source license usually implies GPL, LGPL and BSD/MIT, although there are over 50 on the OSI’s list. I tend to be partial towards the BSD/MIT licenses over (L)GPL, I’ve mentioned this before. For the purposes of this article thought, I’m going to focus on BSD licensed software. Don’t take this as a slam against the GPL, I just wanted to focus on the largest amount of flexibility and ease for developers.

It really is amazing that today you can develop an entire application or service using BSD licensed software up and down the stack. This particular license allows you to modify code and it is up to you if you want to share it. Although it is always recommended to participate in the community, this license doesn’t make it a requirement if you want to distribute modified code (this may be a good or a bad thing depending on your point of view). Thus all types of applications and services can be built upon BSD licensed code, from top to bottom. If you’ve ever tried to figure out which license you have to buy from a company that offers more types of licenses than cars on the road you know that not having to go through that maze can be a great time saver.

So here is what I’m thinking of when I talk about the BSD licensed application stack. At the lowest level we need an operating system. I prefer FreeBSD for this, but there is no reason why NetBSD, OpenBSD or DragonFly BSD couldn’t be used instead. Any of these will provide a complete operating system and depending on your wants or needs you may find one fits you better than the others.

Now that we have an OS (FreeBSD), we’ll need some place to store data that our application or service will be using. SQL databases have grown to fit this need quite well. Because we are focusing on BSD licensed open source software one database really stands above the rest, PostgreSQL. Not only is it a perfect fit for our criteria, it is a great piece database software. PostgreSQL supports many features that users of commercial databases have come to expect (Views, Functions, Schemas, etc).

The way to deploy applications and or services today is on the web. Here again we are fortunate because the most commonly used web server is open source and BSD licensed. The Apache web server is flexible (mod_rewrite anyone?) and powerful.

Finally we’ll need an a programming license to get things done. This one piece of the stack is probably the most difficult to pin down. My pick though would have to be PHP, whose license is close to the BSD license. It is also targeted at for web apps, but I’ve used it for command line applications as well.

The BSD Licensed Application STack (BLAST) is about software that does its job well and has a license that is easy to understand and gives you the ability to get distributed changes to yourself. Activity in the community is optional, but encouraged. For me this means FreeBSD, PostgreSQL, Apache and PHP.

Many of you reading this will be jumping up and down that this is just a rehash of L.A.M.P.. On one level this is true, Linux, Apache, MySQL and PHP/Perl (L.A.M.P.) do satisfy one part of BLAST, open source software that gets the job done. Unfortunately the licensing for some of these products is difficult to understand and in some cases the same license is interpreted in different ways (yes MySQL I’m looking at you).

The components of BLAST may change over time, (perhaps another language besides PHP?) but the intent and abilities will be the same. Good open source code with ability to do what you want with it.

After changing the page layout for Rhyll I added some additional features. First off, there is a new blog just for rhyll.com, where news and updates about the site will be posted. I don’t expect to post too much more about Rhyll here unless I add something that I think is really interesting.

In addition to the blog about Rhyll I’ve added two other specific blogs, for
FreeBSD and PHP. I want to explorer both of these topics in more detail. Additional topic blogs will be created if I feel the urge.

Always nice to start off the day with some good news, FreeBSD 6.0-RELEASE is out today. I’ve got a couple of FreeBSD related projects and posts I hope to get out the door now that 6.0 is here.

It really is amazing how easy it is to put together a system with a terabyte (or more) of storage. For less than $500 you can purchase four Maxtor 300 Gigabyte IDE drives ($118 each from New Egg), putting you well over one terabyte in capacity. If you are looking for something a bit bigger Apple has made that pretty easy with their Xserve RAID unit, up to 7 terabytes for $13,000. I believe hard drive capacity will continue to go up and price per terabyte will drop.

At work I’ve got few systems now with a terabyte or more of storage. While most modern operating systems support filesystems in the terabyte range, their advanced features don’t seem to work well. FreeBSD had no problem running with a terabyte plus filesystem (fs), but snapshots on that fs were pretty much useless because they took forever. Still having to run fsck when problems happen would undoubtedly be equally as painful. Fortunately there has been talk of adding journaling to FreeBSD, hopefully that work will at least take care of the fsck problem. General work on large file systems under FreeBSD is being done as part of project Big Disk.

You might think that a company with more money than many countries (Microsoft) would be able to produce an OS (Windows 2003 Server) that wouldn’t have those sorts of issues, but you’d be wrong. Windows 2003 does fine using a one terabyte plus fs, until want to use things like Shadow Copy and then things start to fall apart rather quickly. From what I’ve been able to determine things work fine for awhile and then shadow copy stops working at some point after the system has been up for awhile. That wouldn’t be so bad if that was the only thing that went wrong, but it isn’t. In Windows 2003 the built in backup software makes use of shadow copy, which makes sense, if shadow copy didn’t roll over and die on large file systems. When ntbackup tries to create a shadow copy and that fails, ntbackup stops and the whole back up fails. Obviously that makes it rather difficult to get a clean backup, at least using ntbackup. But it gets even worse, when ntbackup fails because shadow copy fails it erases all of the old successful snapshots that shadow copy made earlier.

After hunting around for awhile I came across Article 833167 at support.microsoft.com claiming that there is a hotfix that is supposed to fix the problem, but it isn’t publicly available. I have to contact Microsoft support and request the hotfix, after which they’ll determine if they are going to charge me for said support. At some point I’ll have to bite the bullet and ask for the hotfix, but I get the feeling they don’t have a lot of confidence in it yet. How long until SP2 for Windows 2003 comes out? What ever it is I doubt that I can wait that long.

Given how easy and cheap it is to equip a system with a terabyte (or more) of hard drive space it is disappointing that some things don’t work better on that sort of scale.