The Innerworkings of a Geek

Monday, January 25, 2010

Does dreamweaver do ecommerce?"

Oh bless their hearts. I have a client who just launched their first website. They were convinced that drupal was the best way to do subscriptions and manage content for their newsletters. Yay!

However, they had a fairly 'newbie' Drupal designer, coupled with some web designers. It made me realize that many people do not understand the difference between web design and web infrastructure. While they ended up with a drupal frontend, their newsletter is in static html. This means that if anyone knows the URL, they can have free access. It also is a pain to update, with the designer, not the client, needing to make changes to the newsletter. This isn't isolated to small startups. Some very large companies have similar issues, one I remember had their $1000/ea webcasts sitting in a folder, with no access restrictions other than by obscurity. Email that link to a friend, and there goes a potential $1000. Cool huh?

Dreamweaver is a somewhat good tool for creating static html sites. Unfortunately, many stop right there. I call these people web designers, because they are able to design a website. However, there is no business logic, functionality, etc. Its a one-way flow of information, and reminiscent of the internet of 1998. Sadly, 12 years later there is a large portion of people who believe what they see on the screen was printed behind the scenes in dreamweaver. This is fine for the end users (like my clients), but people selling website design services really should know better.

So what is the process for building a website? For me, I use Drupal for every site or web application. It performs remarkably as both a Content Management System (CMS) and Content Framework System (CFS?)... This means I can build intranet sites, little web apps, or major websites and get 80% of the job done before I touch php or sql. It a major time saver. How does the site start?

  1. Design and Specifications (Photoshop/Napkin and Word/Napkin)
  2. Dreamweaver / Coda (HTML design) -- get the design looking good in the browser
  3. Strip the content (at this time is lorem ipsum) from the html, and create a theme based on Fusion. (Usually I'm taking acquia_prosper and customizing it to my needs)
  4. Take the specifications and start converting them to drupal components (Blocks, Panels, Views, Pages, Content Types, node relationships, etc)
  5. Apply the theme from step #3 and start tweaking for drupal components. This is why I usually start with acquia prosper. Things look 'okay' without any additional theming to views or blocks.
  6. Test drupal components and soft launch
  7. Profit!
Note, the above process is fairly general. The specifics about using acquia prosper is good for smaller or quick sites that are inexpensive. This is obviously the process for an enterprise rollout =P

The interesting part above is that dreamweaver is only step 3 of a 6 step process. And for larger sites, steps 4-6 are what take weeks, where steps 1-3 take days.

Why my client asked if 'dreamweaver does security or ecommerce" I had to smile. It doesn't do anything really. Its the paint on the house, doesn't do anything but make the house look good. People don't get a house built and painted without any of the guts, but unfortunately this is what many people do. We have the doors, kitchen appliances, counter-tops, carpet, etc. But we end up without any locks, electricity, plumbing, etc. It looks great on the outside, but is totally useless.

Luckily after a 10 minute conversation, I was able to convince them where dreamweaver fits in the ecosystem of web architecture. And it seems like they understood! Very happy.

So what would I label a drupalist? Web contractor? Web engineer? Web Architect? I like Architect.. as they have to design the plans with all the inner-workings of the house. The designer just puts the finishing touches. It is fun though when people ask what I do, to respond "I'm a drupalist"...

Thursday, December 17, 2009

ORV use vs a Ski area

As I start working on various proposals to setup an ORV park in my hometown, I'm met with some environmental concerns and people wanting to limit or prevent such a place from being setup. But just how devastating are ORV parks? Looking at posts with photos like these make people think that our sport is worse than building cities.
I don't want to compare city or even housing development. I want to look at other recreation activities, specifically one that a lot of conversationalists like: Skiing and snowboarding.

Now what I see here is this: a large clearcut with noisy ski lifts and people going up and down the hill scaring away wildelife. From Onthesnow.com:

# Most downhill skiing takes place on prepared slopes. This involves removing trees and grading the land into suitable runs. In addition, pylons and structures for ski lifts must be installed.

# Due to global warming, snow has become less reliable at lower altitudes and the use of artificial snowmaking has increased. Snowmaking is a water- and energy-intensive process. In some cases, chemicals are added to ensure the formation of snow crystals.

# Similarly, global warming is forcing ski areas to use higher altitudes that are more ecologically sensitive, even glacier areas which are receding due to warming.

# Litter carelessly tossed from lifts or dropped on ski runs does not readily degrade. It takes five years for a cigarette butt to disintegrate.

# Wildlife is disturbed by the initial construction and nightly maintenance of ski trails, and by the daytime skier population.


Now take a look at Reiter foothills. Above IS our impact. We WANT tight trails, we WANT to ruff up bark on trees, and we WANT to make up for that impact. And many of us are willing to pay for the corridors that get impacted by our use. But to claim that Reiter foothills is creating more ecological damage than say, Stevens pass, is just ridiculous. What you call devastation I call 'fun'. On the flip side, what I call clear cut and wildlife impact you call snowboarding.

If you take a look at North Fork, a place once used by 4x4s, dirtbikes, buggies, you can barely recognize its past use. Its been 5 years since the closure, and its amazing what has grown back. Sure you still see bark missing from sides of trees, but they're still alive and thriving. You can't tell me that if a ski area shutdown, that 5 years later it'd look nearly as it did before any ORV use.

Take a look at Mtn bike use. This was an old 4x4 trail, now being used by mtn bikes and hikers:
This is in the same area, taken the same day on an old 4x4 trail not being used by mtn bikes or hikers:

I find it very hypocritical for the environmentalists to be trying to claim ecological damage as a legitimate reason to shut down ORV areas. Our impact is much lower than that of some of their favorite sports.

"The State’s surveys have always shown that non-motorized trail recreationists are the majority, and greatly outnumber ORV recreationists."

For some reason hikers believe they are better than everyone else. Reiter already has an abundance of hiking trails nearby. Wallace falls state park is right there, along with hundreds if not thousands of miles of trails in and around Snohomish and king county. But instead of helping the ORV community make sure runoff doesn't go into the streams, they rally to get our zone of use reduced from ~4000 acres to 1100. Then they take the other bit and use it for their sport.

"DNR should not allow 4x4 “tube buggies” that have killed many trees at Reiter by stripping the bark off the trunks as they squeeze through the forest."

These actions by WTA and the Sierra club will, and already is backfiring. 4x4 users are becoming fed up with both hikers and the process. Many don't care if there are legal areas to wheel, they just find some secret spot and make it their own. This is bad for us and them. no one wins except the rouge wheeler, until they get caught.. then we all loose.
This is already happening, on Nov 27th, 2009 the Everett Herald wrote an article about illegal orv use occurring further east. This is what happens when you don't manage your land properly.

Don't get me wrong, we have impact. But we need the help and understanding from the environmental community to get them to understand what we want to do. We also need their help to make our 'sandbox' not cause external impact. But as long as they fight us, legal places will close, and people will cause more impact in harder to reach, more sensitive areas.

Its time for us to all come together and work for a common goal: provide environmentally sustainable recreation options for ALL recreationalists built on sound scientific and business data.

Friday, October 23, 2009

fs01 server upgrade!

Most of my friends know I have a fairly kickass file server. It started back in early 2005, when Krogebry and I worked together to build a file server for the geekpad. Once we moved to the geekpad 2, it kept growing, and once I took over the server in '07, its seen 10 hard drives and a total of 5.7TB of storage.




Specs:
AMD 3500 w/ ASUS A8N-SLI motherboard
1GB (2x512) DDR 3200
10x HDDs (3x1TB, 4x500GB, 3x250GB)
4U Server Chasis
Antec 650W power supply

It was a pretty beefy box in its day. Heck, for a file server its still a pretty decent system. However, the underlying architecture had some issues:
* A8N motherboard has issues supporting all 8 drives in its SATA ports. max 2TB per 4 ports
* PCI Express was being shared with a hungry 6800GTX Ultra Video card
* No RAID

In the beginning, we went for cheap. Really cheap. We figured we did not want to strip the server in RAID1 (because if one drive died we loose it all) nor did we want to mirror the drives (too much wasted space), and lastly we didn't want to buy all 10 drives at once.

Whats new with this FS01?
RAID5: I decided that it was worth it to invest in a RAID controller and put all the drives in some type of hardware raid setup. The 3ware 9650SE-8LPML card was a great candidate for this. Cost? $470
Server Architecture: While the old AMD was great, it was desktop grade, and it'd be nice to get some server quality stuff in there. But what really tipped the boat was when I got a call from my friend at (redacted =P ) ... he had a development/engineering version of an Intel s3420gplx motherboard AND a Xeon x3470!! Cost? ZERO!


New Power supply: The old Antec power supply is great. Really, it probably would have been fine to keep. But I decided that since that PS has been running for 4.5 years, it'd be a good idea to get something new. Its an Antec TP-750. Cost? $104.99

The issues with 'new stuff'
When my friend mentioned he had a Xeon + MB he had laying around, I thought it was a one or two year old engineering board they had no use for. I never dreamed that it was a BRAND NEW, just released board! And not only is this board new, the whole platform is new! It uses DDR3 1066/1333 ECC ram and has a LGA1156 socket. That caused a few issues:

RAM:
No one in western washington has DDR3 ECC RAM. I called and called, with no luck. I took a 'hail mary' pass, and decided to call up 3D Corporation, based in Bellingham. They no longer do retail sales, but figured it was worth a try. And guess what! 3GB (3x1GB) ECC 1066 DDR3 ram! I wish they had 1333, but I'll take what I can get. besides the difference isn't that bad. When I move to ECC Registered 16 or 32GB ram, then I'll go 1333. Cost? $120

Cooling: So as I'm driving up to Bellingham, I notice something... the proc has no cooler! SHIT! well, perhaps the LGA1156 is like the new Core i7? Nope. Fail. 3D had nothing. There is only one other computer shop in Bellingham that might have a cooler. Nope, they didn't either *(funny story though.. they DID have an LGA1156 ASUS board.. wonder why they had a board and no cooler for it? duurr)*
So... go home and I'm thinking .. crap, I have everything but cooling. and this thing NEEDS cooling. So I figured, what the hell.. another Hail Mary pass: lets call Best buy.

Now at this point, all of you techno-geeks are going to say HAHAHA YAH RIGHT. Best buy has decent computer parts? PLEASE! Well guess what.. they did! Its a Corsair H50, bundled up hybrid water cooling with a radiator fan thingy. WAY overkill, but w/e. it fits so lets get that! Cost: $79

Why Hitachi? In part: because they are the only 7200RPM HDDs out there at 2TB, and they don't have any dreaded reports of bad firmware.
But its a DEATHSTAR! no. Please pull your head out of your 2001 ass. Hitachi is arguably the biggest hard drive manufacturer out there. Most apple laptops ship Hitachi. They are reliable and in my case, cheap. I scored a great deal through Dell.com, making them only $144 and free shipping. You can't beat that for 8 drives! I was looking at the WD EADS 2TB drives, but they cost way too much.

Parts List:




Total Cost: $1465 ($2496 for all components)

Dismantle the old system
There are two things that will be sticking around for the new build: Case and 1TB HDD (OS drive). Since we can hold 10 drives (no CDROM), I figured we'd go for 8 drives+OS drive+CD/BDROM.
What a mess. The SATA and Power cables were everywhere. but it stayed cool.



Once everything was ripped out, did some cleaning and got the new board ready for installation. Damn, I knew this radiator thing was big, but oh I hope it'll fit somewhere!


Completed setup, hooked all the drives up, and the radiator fits surprisingly nicely between the cdrom and the power supply. The bottom is wedged in by cables, the top is held down by a bracket on the case, and its held towards the wall by the thick water cables going to the processor. I may screw it down at some point, but its actually pretty solid in its current position. Also thought about cutting a side exhaust for the radiator, but after a few days of testing, this thing is cool to the touch and the CPU runs just under 35C.


All done! Now comes the daunting task of copying data from the old drives. Since we had no case, I figured the drives would work nicely on the top of the case:


I could only get 6 drives on there at a time. There are 6 SATA ports on the motherboard, plus two from the PCI card taken from the old system. When you subtract the OS drive and CDrom, you get 6 extra spots.
I placed foam on the top of the chassis to keep the vibrations down, and then static bags on top to keep the foam to shock the drives. This all works great, but we're missing a big problem... cooling. After first boot, and let it run for 20 mins, the drives on top were between 40-45C. I don't like the drives over 35C.
So yes, I rednecked it (4wheeler, use what works, right?) by putting a window fan on the front of the case:

Of course, that didn't really help all that much. Lots of airflow, but none of it directed. I needed to create a wind tunnel:

With the cardboard on top, it makes the air flow through the drives, then push out through the sides of the cardboard or go down through the case. Did some benchmarks, and the drives now run a cool 28C.
Of course this was all done because its temporary. The transfer will take a day or two, so I didn't want to put undue stress on drives that have been running 24/7 for 2 to 4 years.

Coming up... Part II -- setting up the RAID, software configuration, etc.

Tuesday, June 16, 2009

Environmentalism, Politics, and 4wheeling

Many of you know I'm an avid 4-wheeler. Its a fun and challenging hobby, and it includes a great community of people.

For some reason, there is growing pressure from environmentalists, hikers, and some politicians to ban ORV use, eliminate or downsize ORV parks, and forbid the creation of new parks. Many believe ORV use is a scourge on our federal and DNR land and must be eliminated.

Well I'm going to point out two flaws this logic:
1) Its not that much worse than hiking and biking trails, especially in comparison to logging, development, and wildfires. 10 miles of 4ft wide mtn bike trails is just as damaging as 5 miles of 8ft wide 4x4 trails
2) By taking away ORV parks, you've just invited wheelers into your own backyard. Instead of wheeling on sanctioned trails, they'll be wheeling on YOUR trails.

Nevertheless, there are groups that won't be happy until 4x4 is illegal. They claim trash, mud and damaged trees as reasons. We respond by having bi-annual cleanup parties where hundreds of people show up to clean up the trails. We also advocate through 4x4 clubs responsible wheeling, going on legitimate trails etc.

But all this effort of closing Reiter is only going to make things worse

Where are the wheelers going to go if Reiter is closed?
Are they going to just sell their $8,000-90,000 rigs and find a new hobby? No.
Are they going to drive hundreds of miles to wheel every weekend? No.
Are they going to find trails off secluded DNR/Forest service road? DING DING DING!

So now, you environmentalists and politicians have moved a contained area, wheeling within a 40,000 acre Park, to a bunch of random 'Fight club-esque' places that no one reveals so they don't get shut down.

Don't believe me? North Fork was the only ORV park in Whatcom county. It was shut down in 2005. Do you think everyone went to canada or walker valley? HA. I'll just say that there is a large contingent of trails wheeled every weekend in Whatcom county, ranging from near Bellingham all the way out to Mt. Baker.

Are work parties to cleanup trash created in these areas? NO
Are there work parties to create culverts and safe water passages? NO
Is there any coordination between clubs to make a manageable trail system that leaves a small footprint on sensitive areas? NO

And there you go, you've just moved a contained group that exists in one area to a fragmented mess spread around western Washington. And the only way to totally eliminate this is by gating off everything. If that happens, then say goodbye to hiking and biking trails, as the trail-heads will be miles from the gate. But heck, even then singletrack can get around gates, it just become unrealistic.

So this is my message to any environmentalist reading this: You WANT to support ORV parks.
When you support ORV parks....
....you're supporting a managed system of trails that are being maintained and patrolled by fellow ORV users and law enforcement.
....you're supporting people who have vested interest in keeping that land clean and enjoyable by all.
....you eliminate or drastically reduce the problem of illegal wheeling in the forest where YOU might be hiking/biking/climbing.

This is my message to politicians who make the policies: Supporting complete ORV parks will help stop illegal dumping, and illegal trail systems
1) Rate trails. To me, it seems like DNR would be more legally sound if they rated trails. If some dumbass gets his rig stuck, flops, etc and there is ample signage to say its difficult, then I'd think they'd be covered in a lawsuit.
2) With rated trails, don't make all of them watered down, if the trails aren't hard for those with 90k buggies, they will go elsewhere.
3) Use the vast 4wheeling community to build, manage, and maintain trails. This is working successfully at walker valley.
4) Create a framework for funding so the 4wd community can allocate funds from their own fundraisers into grants to build trails in ORV parks.
4) get NOVA funds back into the proper places

I find ORV parks to be very similar to Ski resorts, in both environmental impact and trail design. So why can't we get just a few good ORV parks in this area? And why don't we have use fees at ORV parks?

Labels: , , ,

Saturday, March 28, 2009

Earth Hour: Turn your brain off for 60 mins!

I was going to write a Twitter or Facebook quip about earth hour, but I just couldn't fit everything in. So here comes a blog post.

First, not enough people are thinking critically about this. Most people acknowledge that Earth Hour does nothing to conserve energy. Even Edward Norton, the 'US Ambassador' for EA acknowledged this on Larry King tonight.

Then, I'm watching more of Larry King and there is a PSA for Earth Hour. There we have Alanis Morissette ON A PLANE advertising what you need to do for earth hour. wow.

And lastly we have the guy who leads the WWF, who is pushing all these environmental actions, with little scientific research to back it up. None of these three people are scientists, and none of them acknowledge the debate regarding climate change. There is ALWAYS a debate, and making assumptions like they do is similar to 'soft fascism' (This article is definitely libertarian/free market biased, however it makes some good points.)

And all this boils down to Penn and Teller's episode on 'Environmental Hysteria' ... And if you watch it, you'll notice all these 'environmental followers' sign a petition to ban water. Thats right, water.

Climate change is real, no doubt about that. But the way environmentalists are approaching its solution are based on politics, not science. Many advocate turning back the clock on the industrial revolution, or charging for carbon credits, which ultimately harms the poor and middle class. Technology and Science is going to be what gets us out of this. Better solar and wind energy plants, better battery technology, and better transportation technology. The government needs to invest and support companies and universities, not taxing John doe and Corp X. There is a HUGE market for electric cars, and sustainable power, when the technology is good enough to compete. This will happen, I think sooner than later. Until then, if you want to live in an unpowered cave, or drive veggie cars, or spend a whole bunch of money on solar power and make your house energy efficient, kudos. Hybrids don't count ;-)

Think about it this way: Babies are bad for the environment. And if you look at the Earth Hour movement stats, it'd take just one person to not have an extra kid (living in an industrialized nation) to do the same as Earth Hour. Perhaps people should reduce having kids to one or two. That would be more environmentally friendly than almost any other suggestion offered out there.

Earth Hour is about being smug at best, and trying to push dubious science as law at worse. So smarten up and don't follow blindly. Turn your lights on for Earth Hour!

This youtube video pretty much sums up my view: http://www.youtube.com/watch?v=Ifci-W2Cs1o

Wednesday, March 04, 2009

Development environment notes

Here is a good raw list of things to have when developing drupal sites....

Sacha from IBM -- http://bit.ly/drupal25

Spend the time to make a tool, if it make sense
Read editor documentation
Some browser plugins Tamper Data, iMacros, Drupal for Firebug (plugin and drupal module), trace module, drubuntu (for multisite is very useful)

Simpletest! I've got to research simpletest more

source code management - using svn, git, etc
--check out the whole source tree
--check in clean source for third party
--organize the sites/modules dir for development, etc

use update functions in the custom module install code.
create svn branches for production and dev
use make or ant to build your system
* sql --- connect db
* backup --backup db
* restore -- restore db
* clearcache -- clear cache
* tags -- rebuild VI tags
* doc -- update doxygen docs
* test -- run project tests
* parse settings.php file to get variables

Use the drupal shell! http://www.drupal.org/project/drush
vimperitor -- allows vim shortcuts for firefox
Use Virtual document root - apache can be the SVN checkout
http://httpd.apache.org/docs/2.2/mod/mod_vhost_alias.html#virtualdocumentroot
unfuddle
Write install profiles for projects
Aegir

Friday, February 27, 2009

LTSP vs Hybrid (Diskless node) clients

This week, we rolled out our new deployment of linux workstations. These systems are running SLED11 RC4, with Firefox, OpenOffice, etc. Featurewise they are the same as our OpenSUSE 11 LTSP machines. However, they're now Hybrid orDiskless nodes

Some history....
When I got into deployment of LTSP machines, I thought thin-computing would be pretty straightforward. At the beginning it was. We had only a few users, they were not power users (only Firefox, OpenOffice) However, the more users, the more people wanted to use them as normal workstations. People in the LTSP channels boasted how easy they were to administer, the reliability of just servers to maintain, etc, etc. I was excited when I got my first crack at a deployment. But would it match up to a normal workstation? In short the answer was no.

Why did we choose LTSP?

1) Reduce Costs
By having two servers operating for 40 users, we should be able to have 40 thin clients priced below $300/CPU. When I approached dell for a thin client terminal, they wanted $625 for just the CPU! I said no thanks, and instead went the custom route. We spent between $275-$225/ea CPU, plus ~$200 for Samsung 19" LCDs. Prices have come down on LCDs since then.
2) Reduce management
Two servers are much easier to update than 40 clients.
3) Workstation security
Data is always stored on redundant, backed up servers; not the client. The client also doesn't have local root and cannot install local apps, so the machine won't be infected with viruses.
4) Reliability
If the local machine crashed, it can easily be swapped out for another machine. Since the server is the only system being maintained, it should be more reliable.

What went wrong with thin-client computing for us? I wrote about it more here: http://www.fcdnet.org/japerry/2008/12/good-bye-ltsp.html

Why it failed:
1) They're slow.. painfully slow. You'd think with 2 8-core, 8GB, Raid 0 servers you'd have fast clients. When compiling, yah they're fast. Otherwise, they are really slow. Monika mentions that her work also uses thin-clients, only the windows variant. They too are slow and have reliability problems.
2) Unreliable. If the server has issues, all users are affected. We did everything possible to keep them running, but we hit a major unexpected roadblock here: Many apps are just not tested or designed to run in a multi-user environment. Especially file/network protocols. Novfs, NFS, NSS, SMB all one issue or another with multiple users on one machine. And with other apps we find 'odd' issues that just don't occur on single machines.

So what about a diskless node solution? Keep the drive remote and treat the client similar to a liveCD.....
KIWI-LTSP project is much more than LTSP. It also is for image automation. although still in beta, its used in many production processes. SuSE releases its Live/Install CD/DVDs using the kiwi imaging system. Suse just announced SuSE studio, which is a web based service to create custom disk and vmware images, essentially a front-end to the kiwi imaging system. But there are other things you can do with kiwi as well, including boot strapping a NFS, NBD, or AoE squashfs system.

Our Solution: AoE diskless nodes, using KIWI.
Cyberorg recommended I take a look at AoE support in KIWI. Work was done recently to allow AoE exports to mount in kiwi. Ata over Ethernet. The nice thing about AoE is that its performance is almost that of iSCSI. The disadvantage (some would say) is that you can only export it on the local lan, since its not routable. for our situation, this is fine, and good because it provides an extra layer of security. Our hybrid solution sits behind its own VLAN.

The other major advantage of AoE and KIWI is ease of image updates. Once the kiwi netboot initrd is setup, you can change your images without needing to rebuild kiwi. For our deployment, I have a dedicated hard drive for images. This allows us to fully test (IE reboot into) a snapshot of our image. Once testing is done, I reboot into my deployment machine, run a few custom scripts to sanitize /tmp, /var/log, /etc (get rid of udev, fstab, etc), and export the whole drive as a squashfs.

 mksquashfs /mnt/staging /srv/kiwi-images/`cat /mnt/staging/etc/suse-release`-`cat /mnt/staging/etc/kiwi-label`-`cat /srv/kiwi-images/version-incr`.img 

Once this is done, i can deploy the new image.
losetup -a

losetup -d /dev/loopN (where N is the loop devices already mounted

losetup -v -f /srv/kiwi-images/`cat /mnt/staging/etc/suse-release`-`cat /mnt/staging/etc/kiwi-label`-`cat /srv/kiwi-images/version-incr`.img

losetup -a // verify what loop device the image is deployed on
vbladed 0 1 eth1 /dev/loopN


One drawback to the process above is the need for everyone to shutdown their systems before the image can be deployed. There is a workaround to this. If you don't delete the previous image, you can change the pxeboot.cfg file to indicate the next loop device (probably /dev/loop1). Then, whenever someone reboots, they'll get the newest image, but currently logged in users aren't affected. Depending on your user base, you may want to incorporate a cron job to forcibly shutdown/reboot the machine (for example saturday morning, 3am). Since these are linux machines, some of our LTSP clients have had an uptime near 60 days. If you decide to deploy an image, and then delete the old one a month later, you may find two or three users calling the helpdesk wondering why they get 'cannot execute file: input/output error' and cannot reboot their machine without hard shutting it off.

Another solution is to create one base image, and perform updates entirely from scripts. For servers I could see this being useful, but for clients its a little more sketchy. Right now the image deployment allows a user to be at the login screen exactly one minute after they press the power button (and 20 seconds of that is before pxeboot even starts!) If we ran zypper to install custom packages, the boot time would increase considerably. So we install all needed packages on the system, and use configuration scripts to decide if something should start on boot.

Custom configurations
In LTSP, we could deploy the lts.conf file, which would setup sound, printers, and other special devices an individual user may have. Since we're no longer using LTSP, I needed to come up with a different solution. I created a script that gets called on boot that copies or appends files based on the hostname. This allowed me to get custom printer.conf files for users, which would append to the company-wide printers already installed on the image.

Home directories, stateful data storage
We use NFS to mount the /home directory from what used to be one of our LTSP servers. This allows users to keep all their files as they were. We decided against using NSS or Novell NCP for home directories because of input/output errors. /tmp is currently stored on the squashfs image, as are the logs. However, I think this could potentially pose a problem, since it could fill up ram. I'll probably make a new nfs share for /tmp.

Thin-Client specs:
For our new deployment, the thin-clients remain mostly the same. They still do not have local storage. When we went with LTSP clients, I wanted to go overboard on the specs. Many boast that 'Pentium II' CPUs w/32mb ram and 8mb video works great with LTSP. Since crummy small LTSP clients sold for the same as my desktops, I went the desktop option. What do they have in them?
AMD 3800 or AMDx2 4200 CPU
256 - 1GB RAM
CD/DVD ROM/RW drive
NVIDIA 6050/7150 onboard video -or- 9400GT
GigE ethernet
Turning thin-clients to the hybrid-clients only required more ram. Since ram comes a dime a dozen, this upgrade was very inexpensive. We found that systems running 256MB RAM could not display GDM, those with 512MB could run only one app at a time, and 1GB works ok. We're doing some exhaustive testing this week to see if 1GB is okay. If not, we will update the machines to 2GB. Guess how much it costs per machine to goto 2GB of ram? For 38 machines (after re-purposing the ram we have), it cost $360 from newegg. Thats right, $20/ea.

Performance improvements:
The first thing we looked at was video performance. We thought at first that nvidia 6050 cards were just crap and no better than intel or sis onboard graphics. Apparently LTSP was the problem. Tests with simple glxgears show a 200% improvement in frame rate, 9400GT got a 250% boost in performance.
Thunderbird used to render calendars poorly. So poorly that our power users had to use windows until we got this system up and running. Now? thunderbird displays every thing instantly.
QTIplot with graphs rendering. It would bring the LTSP clients to the brink of locking up (and many times it would lock up). Now? qtiplot renders perfectly.

In summary
LTSP was a great idea when the servers were less than the price difference of thin-client machines vs thick clients. All other benefits are solved with the hybrid-client solution. And since commodity computer parts are as cheap as thin-clients, there is no benefit of using LTSP.
If you want the benefits of LTSP without the drawbacks, take a look at the hybrid-client solution. Its cheaper, faster, and more reliable than thin-client computing.