Archive for the ‘Tech’ Category

Adempiere now available

Sunday, October 26th, 2008

Adempiere LogoAdemipere 3.52 (and 3.51, FWIW) are now available.

They are running on either Ubuntu Hardy 8.04 LTS, or Ubuntu Intrepid 8.10, your choice.

Both are running on PostgreSQL 8.3 and the latest PLJava.

All our VMs have RDP access (via xrdp) built in on request, and the usual KVM VNC, and SSH access as well.

The ZK Web UI on 3.52 is a big step forward from 3.51, and is pretty impressive!

We can also help you integrate your Adempiere backend with other SAAS  Services (aka ESB / Enterpise Service Bus),such as Jitterbit integration / glue software http://www.jitterbit.com/Product/screenshots.php, also see here:http://jeff-davis.blogspot.com/2007/05/jitterbit-screencast-demonstration.html

http://en.wikipedia.org/wiki/Enterprise_service_bus

Also, a great find: daily Adempiere binaries and WebStore demo – (it’s .jsp, though :-/ ):

http://testadempiere.com/
Test ADempiere Web Components

http://testadempiere.com/webui/

And Workspace by Smilehouse – Finnish Adempiere => eCommerce bridge / transformation using OpenSyncro

http://workspace.smilehouse.com/overview.shtml

WS Open Interface API

http://www.opensyncro.com

As always, we can help you integrate / launch / evaluate these technologies, and plan your rollout strategy for them and Adempiere.

We can also just help you get started.

j

Virt IO Performance

Saturday, October 4th, 2008

banner_kvm_forum_2008

One of the latest cool new virtualization features KVM offers, is “VirtIO”, which is paravirtualized networking – a fancy way of saying “Really fast networking that runs at or near native host speed”.

Although the tun/tap based networking we’ve been using for over a year has proven itself to be very fast and reliable, we’ve done some experimenting with VirtIO after attending this June’s KVM Forum in Napa.

We’ve applied some benchmarks from the KVM virtio instructions, here:

Using VirtIO NIC – Kernel based Virtual Machine

and came up with about 40% higher than their results, guest-to-host, even with no virtio:

Ubuntu Intrepid Ibex client, v.sep30, no virtio:

sysadmin@unconf-ubu810-oct3:~$ sudo iperf -c librehost.com -w 65536 -p 12345 -t 60
————————————————————
Client connecting to librehost.com, TCP port 12345
TCP window size:   128 KByte (WARNING: requested 64.0 KByte)
————————————————————
[  3] local 216.218.243.155 port 60365 connected with 216.218.243.130 port 12345
[ ID] Interval       Transfer     Bandwidth
[  3]  0.0-60.0 sec    995 MBytes    139 Mbits/sec
sysadmin@unconf-ubu810-oct3:~$

Host/server:

root@librehost-h64:~# iperf -s -w 65536 -p 12345 -I 5
WARNING: option -I is not valid for server mode
iperf: ignoring extra argument — 5
————————————————————
Server listening on TCP port 12345
TCP window size:   128 KByte (WARNING: requested 64.0 KByte)
————————————————————
[  4] local 216.218.243.130 port 12345 connected with 216.218.243.155 port 60365
[  4]  0.0-60.0 sec    995 MBytes    139 Mbits/sec
root@librehost-h64:~#

After virtio installed & configured, showing ~5x speedup:

sysadmin@unconf-ubu810-oct3:~$ sudo iperf -c librehost.com -w 65536 -p 12345 -t 60
————————————————————
Client connecting to librehost.com, TCP port 12345
TCP window size:   128 KByte (WARNING: requested 64.0 KByte)
————————————————————
[  3] local 216.218.243.155 port 36958 connected with 216.218.243.130 port 12345
[ ID] Interval       Transfer     Bandwidth
[  3]  0.0-60.0 sec  3.90 GBytes    559 Mbits/sec
sysadmin@unconf-ubu810-oct3:~$

And on the server/host:

root@librehost-h64:/vm/155# iperf -s -w 65536 -p 12345 -I 5
WARNING: option -I is not valid for server mode
iperf: ignoring extra argument — 5
————————————————————
Server listening on TCP port 12345
TCP window size:   128 KByte (WARNING: requested 64.0 KByte)
————————————————————
[  4] local 216.218.243.130 port 12345 connected with 216.218.243.155 port 36958
[  4]  0.0-60.0 sec  3.90 GBytes    559 Mbits/sec

In conclusion, with this 5X speedup, all our Ubuntu Intrepid Ibex-based vMachines will offer virtio installed soon; likely by the time Ibex is released.

Ask for it and we’ll be sure to install it by hand on your VM if so desired.

j

Review of Django Database Migration Tools

Monday, September 1st, 2008

Update 12/1/08 –

South Logo - Django MigrationsSee comments below, and look for a new blog post from us on a new player which we’ve recently discovered – south:

http://south.aeracode.org/

They even have their own review of the other players, here:

http://south.aeracode.org/wiki/Alternatives

We will post a review on this soon.

Preamble

Following is a review I did recently for one of our Django clients, of six or so of the various tools available to handle database migrations – much like the “Migrations” facilities built in to Ruby on Rails.This list is by no means complete; as there are a lot of players our there – too many – so many, in fact, that no clear winner has emerged.  Read on.

The Reviews

The issue is how to handle schema migration and schema version control in a way that integrates well with the usual Django development workflow.

Here is an excellent Django-centric view on the topic, with proposed solutions and some of the projects currently available:

http://code.djangoproject.com/wiki/SchemaEvolution

I have checked all the references in the article, and some further ones I have found –
here are the candidates for use, and my evaluations:

Django Evolutions

http://code.google.com/p/django-evolution/

  • Mature, active, likely candidate for future Django inclusion
  • Postgres support the strongest, MySQL weakest
  • Establishes a baseline, rather than introspects – stable, conservative approach
  • Creates two new tables for the baseline – django_project_version, django_evolution
  • Does not help for our current situation
  • Builds on RoR / ActiveRecord? ideas, takes one step further

Django Schema Evolution

http://code.google.com/p/django-schemaevolution/

  • Based on the above article/link & observations
  • actually does introspections, too
  • MySQL support the strongest
  • Would work well for us, except it faults on current Django SVN release we are using (I filed the issue on Google code for the project).

deseb – Django External Schema Evolution Branch

http://code.google.com/p/deseb/

  • Appears to be a current project, with a very active community
  • this appears to be a fork of the schema_evolution project, and is also based on the article/link above
  • it uses a different approach, doing signatures of the SQL, and
  • it does a virtual ‘diff’ on the Model vs the schema –
  • almost like a db ‘lint’ utility – I will post the output below or next post
  • can optionally generate schema migration scripts to be used in version control and managed deployment scenarios
  • at the very least, this is a very useful utility to compare existing Models/schema with the manage.py sqlevolve command (see output below)

BEGIN;
ALTER TABLE `blog_categories` MODIFY COLUMN `id` integer AUTO_INCREMENT;
ALTER TABLE `blog_posts` MODIFY COLUMN `id` integer AUTO_INCREMENT;

ALTER TABLE `tagging_tag` MODIFY COLUMN `id` integer AUTO_INCREMENT;
[lots of similar auto-increment mods clipped for brevity]
ALTER TABLE `django_redirect` MODIFY COLUMN `id` integer AUTO_INCREMENT;
ALTER TABLE `django_session` MODIFY COLUMN `session_key` varchar(40);
ALTER TABLE `django_site` MODIFY COLUMN `id` integer AUTO_INCREMENT;
ALTER TABLE `robots_url` MODIFY COLUMN `id` integer AUTO_INCREMENT;
ALTER TABLE `robots_rule` MODIFY COLUMN `id` integer AUTO_INCREMENT;
ALTER TABLE `wtp_user_profile` MODIFY COLUMN `user_id` integer;
— warning: the following may cause data loss
ALTER TABLE `wtp_user_profile` DROP COLUMN `is_subscribed`;
— end warning
ALTER TABLE `wtp_user_review` MODIFY COLUMN `id` integer AUTO_INCREMENT;
[lots of similar auto-increment mods clipped for brevity]
ALTER TABLE `wtp_product` MODIFY COLUMN `id` integer AUTO_INCREMENT;
UPDATE `wtp_product` SET `is_new_image_url` = ‘t’ WHERE `is_new_image_url` IS NULL;
ALTER TABLE `wtp_product` MODIFY COLUMN `is_new_image_url` bool NOT NULL;
ALTER TABLE `wtp_genre` MODIFY COLUMN `id` integer AUTO_INCREMENT;
[lots of similar auto-increment mods clipped for brevity]
ALTER TABLE `wtp_editorial_review` MODIFY COLUMN `product_id` integer;
UPDATE `wtp_editorial_review` SET `is_published` = ‘f’ WHERE `is_published` IS NULL;
ALTER TABLE `wtp_editorial_review` MODIFY COLUMN `is_published` bool NOT NULL;
ALTER TABLE `wtp_featured_article` MODIFY COLUMN `id` integer AUTO_INCREMENT;
ALTER TABLE `wtp_user_review_feature` MODIFY COLUMN `id` integer AUTO_INCREMENT;
ALTER TABLE `wtp_user_review_feature_rate` MODIFY COLUMN `id` integer AUTO_INCREMENT;
ALTER TABLE `wtp_screenshot_meta` MODIFY COLUMN `product_id` integer;
ALTER TABLE `wtp_screenshot_image` MODIFY COLUMN `id` integer AUTO_INCREMENT;
ALTER TABLE `wtp_product_video` MODIFY COLUMN `product_id` integer;
[lots of similar auto-increment mods clipped for brevity]
COMMIT;

yadsel – Yet Another Database Schema Evolution Library

(Formerly dbMigrations)

http://code.google.com/p/yadsel/

Reasonably active project, last updated April 2008, some / sparse community participation –

  • Bills itself as an “Active Record Migrations like project”
  • Full and wide db support, and Django support
  • Full schema inspection, ddl/uml support, etc
  • Overkill for what we need
  • Lots of docs, but no ‘Getting Started’ guide, as is customary, unlike (All1) the other candidates
  • Not installed and evaluated

dupdater.py — Django project evolution utility

http://www.methods.co.nz/django/dupdater.html

Basically a version management & deployment tool, to give you the very basics of the RoR ActiveRecord? functionality – versioned back-and-forth scripts.

  • Simple python utility script, no Django installation required
  • Does one thing, and does it well – no introspection, etc
  • Active, last updated Apr 08
  • Immune to the Django 1.0 migration problems
  • We should evaluate this further, it looks good

Great for:

  • Keeping track of your SQL db alterations
  • Keeping them under version control
  • Allowing them to move forward and backward
  • Managing deployment in various environments – Production, Staging, Development, Sandbox.

This is simple and elegant, we may want to use this along with deseb for ‘db lint’ and to help generate the updates.

dbmigration — Simple schema migration for Django

http://www.aswmc.com/dbmigration/

Currently at v0.4, and about a a year old – does not appear to have a lot of traction, although it’s widely quoted and linked to.

  • Also patterns itself after the back-and-forth AR-style scripts
  • Uses a ‘migrations’ directory per app (rather than per project, like AR / RoR)
  • Decries auto-update integrated migration tools like Django-evolution and Django-schemaevolution
  • Supports migrations written in SQL or Python.
  • Not installed or evaluated

Recommendations and Conclusions

  • Use deseb for its ‘DB Lint’ functionality (at the very least, on dev)
  • (Possibly de-fang its update capabilities)
  • Evaluate and likely use dupdater for deployment & SQL version management
  • Install/Leave django-migrations installed, to see how it tracks the baseline
  • Keep a close eye on django-migrations moving forward, to see if we can start using it, and how it tracks the baseline

j


UPDATE 09/02/08 10:03:37 PDT dupdater findings: ¶

  • dupdater is at version 0.6, and feels at least this rough-around-the-edges
  • After using it on sandbox for a while, I can’t recommend it at this time.

However, the workflow that this tries to mimic/achieve, ie, a directory of changes, numbered, which are applied in order, is a good one – We can just do this manually with our /sql/ subdirs.

There is even some support for auto-application of sqls in the app /sql/ subdirs, by the manage.py interface, if we ever choose to use it.

Updated Recommendations and Conclusions ¶

  • Use deseb for its ‘DB Lint’ functionality (at the very least, on dev & sandbox)
  • (Possibly de-fang its update capabilities)
  • Use a /sql/ directory of numbered .sql files for the db updates
  • Install/Leave django-migrations installed (at least on sandbox)
  • Keep a close eye on django-migrations moving forward, to evaluate when to start using it and how it tracks the baseline

j


UPDATE 09/07/08 12:14:24 PDT

FYI, as of 9/7/08, deseb works fine for us in our current environment, but has not yet been updated to Django 1.0.  You will have to remove it if you are migrating to Django 1.0+.

j

Now on Django 1.0b and Satchmo 0.8pre

Monday, August 25th, 2008

django-logo-positiveOK, after some serious work and refactoring, we are now up and running on Django 1.0b and Satchmo 0.8pre, as of today –

The new shop and Django features are quite impressive, and include many things we used to have to do ourselves.

You will see us gradually putting our Satchmo eCommerce ‘shop’ back together over the next few weeks to be back up to full functionality.

As always, you may reach us for sales and support via email.

j

Now Upgrading to Satchmo 0.7!

Sunday, July 6th, 2008

satchmo-front
Please pardon our dust (please order by email or phone!) as we reimplement portions of our eCommerce engine to use the new updated Satchmo Release, 0.7 (we are using SVN trunk).

Previously, we were on the Satchmo 0.6 release (and we implemented the TrustCommerce payment engine for the project, for that release).

We hope to have the eCommerce signup capability back up and fully functional within a short time.

In the meantime, we can accept your orders and support emails through our usual email and phone channels.

Joe

Uptime

Sunday, June 1st, 2008

Uptime Collection

This post serves as a collection of our notable uptimes – for several things:

  • Physical servers
  • VMs / Virtual Servers
  • Application platforms that keep uptimes, such as Zope

We will also be posting notable uptimes as comments to the post so that you can get an idea of how we do things.

Good Example [Updated! March 18, 2010]

Here’s one of our primary webservers, running Debian, installed in 2005 prior to our adopting Virtualization across the board in our product line:

webserver:~# uptime
16:26:01 up 1534 days, 10 min,  7 users,  load average: 0.16, 0.32, 1.04
webserver:~# date
Thu Mar 18 16:26:11 PDT 2010
webserver:~#

Yes, that’s right – 4.2+ years.

One of our Physical VM hosts

Here’s our current uptime, just prior to our kernel upgrade which will force a reboot:

root@librehost-h64:~/libre# uptime
17:27:36 up 96 days, 18:36,  4 users,  load average: 0.16, 0.26, 0.27
root@librehost-h64:~/libre#

After this upgrade we’ll be on production/release post-SSH-key Ubuntu Hardy 8.04 LTS.

j

Ubuntu 8.04 Hardy Now Available!

Wednesday, April 30th, 2008

hardyheron3-mediumThe latest release of Ubuntu is out, Hardy Heron 8.04 LTS, and it’s gotten Great Reviews so far –

And we now have our vMachine available and ready to go, with the usual package buildouts of our favorite platforms.

We are finalizing the installation scripts and packages, but currently available are Rails 2.02 and Django 0.97-SVN with others forthcoming – watch this space for details on finalized package lists.  (Our previous postings on Django and Rails indicate some of the usual choices)

Also, as always, we use the KVM virtualization technology, which provides a number of technological advantages over the other v12n solutions.

Our vMachine is based on the minimal footprint of the Ubuntu JeOS release, which is now Fully optimized for KVM, including VirtIO, according to Canonical/Ubuntu –  we have a mere 700MB basic footprint for the guest, with a full array of services including ssh.

With your favorite stack from us, your vMachine will still typically be under 900MB, even with xrdp  for your remote desktop XWindows access!

We can host your site / solution and provide any technical assistance and consulting help which you may need.

New: We can also host your desktop!

In addition to the usual VNC (and of course SSH offerings, including X) We offer RDP via XRDP, and we are working on a fully open-source version of the GPLed NX NoMachine technology – and we also hope to be able to offer Qumranet’s (A cool vendor) SolidICE  technology – watch for an announcement.

3 Flavors of Ubuntu

Friday, April 4th, 2008

ubuntu-worldWe always offer (at least) three flavors of Ubuntu:

  • Long-Term-Support (Dapper, 6.06, as of this writing)
  • Current (Gutsy as of this writing)
  • New alpha/prerelease (Hardy Heron 8.04 Alpha 4 as of this writing)

Once Hardy is released, these will shift one-to-the-right, along with the next pre-release (Intrepid Ibex).

Please also note that you can always get a the 64-bit amd64 version on request, although the 32-bit i386 version is the default.

j

Ruby on Rails 2.02 / Ubu7.10 Now Available!

Sunday, March 9th, 2008

Railsters,

We have launched an interim release of the latest Ruby on Rails 2.02 platform on Ubuntu Gutsy 7.10.

Although our longer-term plan is to provide Rails 2.02 on the more recent Ubuntu Hardy 8.04 LTS / Debian-based platform, this release supports the latest Rails with the usual plugins and support you’ve come to expect from Libre Hosting.

We follow the recommended installation procedure for Ubuntu with Rails, found here: https://help.ubuntu.com/community/RubyOnRails

We also include many defaults and well-chosen packages by us as well as favorites within the community, such as Mongrel, MySQL, OpenSSL, and so forth.

Here’s the way we installed it – let’s start with a few useful utilities, ssh, mysql, lighty, followed by basic ruby and gems only:

apt-get update
apt-get -y install wget mc ssh curl lynx zip unzip arj rar unrar rsync rzip build-essential
apt-get -y install mysql-server libmysqlclient15-dev lighttpd
apt-get -y install ruby rdoc irb libyaml-ruby libzlib-ruby ri libopenssl-ruby ruby1.8-dev
apt-get -y install rubygems

Note that we only want the basic ruby language install – everything else will be maintained by gems, including rails, so we install only rubygems itself from Debian/Ubuntu, and install Rails (and any/all other Ruby dependencies) directly via gems. Be sure you don’t step on gems by using apt-get to install ruby packages! Always use gems.

Also, note that the update –system updates the libraries to gems’ dependency needs, not Ubuntu’s – so now further ruby (and Rails-related) packages will install consistently with gems.

Here we go:

gem update –system
gem install rails -y –include-dependencies
gem install mongrel

apt-get -y install libmysql-ruby
gem install mysql

apt-get -y install imagemagick libmagick9-dev
gem install rmagick

gem install capistrano
gem install puppet

NOTE: RMagick does not install due to version conflicts with the older version of ImageMagick found in Ubuntu7.10. We will address this issue shortly.

PostgreSQL and/or SQLite3 can also be installed on request – as well as GeoKit, and the Rails examples.

Thanks,
LH Staff

Launching RDP & VNC, NX

Sunday, March 2nd, 2008

The Libre Hosting vPanel offers links to launch various viewer programs automatically, including the universally available VNC (Virtual Network Computing, http://en.wikipedia.org/wiki/VNC).

Although there is a Java-based VNC viewer available in the vPanel as well, which you can simply launch from within your browser, we also offer the ability to launch your local, native viewer of choice, and offer support for VNC, RDP (both the Microsoft Remote Desktop Connection, or RDC, as well as the Linux-based RDesktop and TSClient programs), and even No-Machine NX and FreeNX.

To simply connect to your vMachine with your existing VNC client manually, just point your vnc client to librehost.com, and connect to your vnc port number, which can be found on your vPanel Status page, right at the top, in the General Information section, under Ports.

We are also listening to any suggestions you, the reader, might make with respect to new protocols and viewers to support – we are aware of the new Vinaigre client for Linux, and are looking forward to some more technical information on supporting the SPICE protocol from Qumranet, the authors of KVM, which we use for our virtualization. We will also be happy to implement and support Citrix – Contact us today to discuss your Citrix-based VM access needs.

We also may choose to enhance the NX protocol, and incorporate it natively into our infrastructure – this is a pet project of management – let us know if this is something that you’d like to see happen in the near future.

Now, on to the cookbook-style HOWTO:

1) The “Launch” links work much like “media” links – they merely launch your viewer and contain the information you need to pass to it to find your vMachine and display it properly, and the file type / file extension (and MIME type) are used in order for your browser to determine the correct viewer program to launch on your local computer, in order to work and properly view your vMachine. So some clients can be launched directly from the link, and for certain other clients and platforms we need to use a small script to then launch the actual viewer, with the correct configuration and file.

2) Your viewer program (such as your client and platform’s flavor of VNC, RDP, NX, etc) must be installed on your client system that you’re browsing from.

3) When you click on the launch link, it must correctly launch your viewer. This is why you need to associate it properly when you click on it for the first time.

4) We have helper scripts for some/most OSes and clients, here:

Windows

Click on the ‘rdp’ link next to your vMachine Thumbnail –

This will open a dialog box in Windows which shows it is a “Remote Desktop Connection” and will ask you if you want to open it (possibly “with RDP File”). Click OK. and the built-in Windows RDP client will open and allow you to access your vMachine (be it a Windows, Linux, or any other vMachine which has been RDP-enabled!).

Tested with Windows XP, Windows 2000, Windows NT4. Add your experiences as comments or forum postings and and we will update this list.

Linux

5) For VNC, cut/paste this into /usr/local/bin/launchvnc.sh :

#!/bin/sh
echo `cat $1` >/tmp/launchvnc.log # create log if desired
vncviewer `cat $1`

And associate clicking on .vnc files (or the application/x-vnc mime-type) with the above script, by clicking on one of the ‘vnc’ links in the vPanel, and following your dialogs to associate it with the script, above.

6) For RDesktop, cut/paste this into /usr/local/bin/launchrdesktop.sh :

#!/bin/sh
echo `cat $1` >/tmp/launchrdt.log # create log if desired
rdesktop `cat $1`

Then, associate clicking on .rdt files (or the application/x-rdt mime-type) with the above script, by clicking on one of the ‘rdt’ links in the vPanel, and follow the dialogs to associate it with the script, above.

Apple Mac

TBD.

Please don’t hesitate to add your questions, comments, or experiences as blog comments or forum postings and and we will update this list.

Cheers,
Joe