Fixing Yum dependency problems on RedHat Linux and CentOS

Yum update fails

I see the question of failed dependencies come up on forums quite a bit. Users post errors like:

Missing Dependency: bind-utils = 9.3.4-10.P1.el5_3.1 is needed bind-sdb-9.3.4-10.P1.el5_3.1

or

ImageMagick-perl-6.2.8.0-4.el5_1.1.x86_64 from installed has depsolving problems
--> Missing Dependency: /usr/lib64/perl5/vendor_perl/5.8.8/x86_64-linux-thread-multi is needed by package ImageMagick-perl-6.2.8.0-4.el5_1.1.x86_64

or

httpd-2.2.3-22.el5.centos.2.x86_64 from updates has depsolving problems
--> Missing Dependency: /usr/share/magic.mime is needed by package httpd-2.2.3-22.el5.centos.2.x86_64 (updates)

In all these cases, the solution is to clean yum:
yum clean all

References


Creating a User Defined Function (UDF) in Java for IBM DB2 9.7

Intro

One of the great features of DB2 is that it can be extended with custom code using SQL, C/C++, Java and COBOL. One of the great new features added to DB2 in 9.7 is the ability to run native Oracle PL/SQL code, that capability opens up a world of possibilities, and at the same time, the need to extend DB2 with functions that might have been available on the Oracle install and now need to be replicated on DB2. In this short article I describe how to extend DB2 9.7 with Java UDFs.

Prerequisites

  • Check that Java is properly installed and configured on DB2:
    db2 get dbm config|grep JDK

    The output should show the path to the JDK installation in DB2. If that isn’t the case, you need to consult the DB2 install manual and make sure Java is installed and configured for DB2.
  • Make sure that your user has access to java and javac, and that it is the same version or compatible with DB2’s JDK
  • Set the database manager parameter KEEPFENCED to NO, so that the UDF will be reloaded in each call.

Writing the test code

  • Use your favorite text editor or IDE and create a file called udftest.java with the sample code:
    public class udftest {
            public static String udftest(String domainName) {
                    return "Domain Name: " + domainName;
            }
    }
  • Compile the java program:
    javac udftest.java
  • Create a jar file:
    jar cvf udftest.jar udftest.class
  • Register the jar file with DB2:
    db2 "CALL sqlj.install_jar('file:/home/bnagy/dev/java/udf/udftest.jar','UDFTESTJAR')"

Creating the UDF in DB2

  • Use your favorite text editor or IDE and create a file called udftest-create.sql with the sample code:
    CONNECT TO SAMPLE;
    CREATE FUNCTION udftest(VARCHAR(255)) RETURNS VARCHAR(255)
    FENCED
    EXTERNAL NAME 'UDFTESTJAR:udftest.udftest'
    NOT VARIANT NO SQL PARAMETER STYLE java LANGUAGE java
    NO EXTERNAL ACTION;
    CONNECT RESET;
  • Run the scritpt in DB2:
    db2 -tvf udftest-create.sql

Testing

To check that the function works, run the following command on the command line:
db2 "select udftest('test.com') from sysibm.sysdummy1"

You should get a regular SELECT result set with the expected values. If something goes wrong, you can look at the db2diag.log that will contain any Java stack trace that would be generated.

References


Fiber Channel (FC / SAN) Performance Tuning on AIX 6.1

Introduction

In large data-warehousing applications, the efficiency of storage systems is critical. With the same hardware and software release, we have seen queries that never terminate (i.e. still runs after a week) under one configuration to be able to finish within hours after optimizations.

Of course in the case of a database like DB2, there are many other factors to consider, and in this article, I’m focusing on the OS level tuning advice from my friend Ben.

About FSCSI

When using a SAN over FC, it is possible to get close to wire speed, as the FSCSI protocol has very low overhead. It is usually more the nature of the I/O that slows down performance. Synchronous I/O is going to be slower – more on that later.

Testing I/O

A good way to test out tis by compiling up the latest version of IOtest on AIX:
ftp://ftp.soliddata.com/iotest/
Use it on a file or a junk raw LV while supplying it with various parameters.

Things you will usually see:

  • synchronous I/O is going to be far slower than you would expect no matter what you do. (synchronous I/O is produced by commands like dd, cp, mv, cat, etc. as well as by databases and applications that do not use AIO (Unidata database, websphere, apache, etc. all use Synchronous I/O)
  • The more writes you have, the slower you will go. Most SAN disk manufacturers design their arrays (and even their spindles) to handle 80% read and 20% write optimally- so that even if you have a large amount of disk cache, you will fill up the allotted “slot” for writes very quickly and ultimately be limited by the ability of the back-end disks to absorb writes.
  • The smaller the I/O sizes are, the lower your throughput
  • The fewer process threads performing I/O, the lower your throughput – the more process threads you are running, the better the saturation and I/O consolidation, so the better your throughput (but only up to a point)

Detecting a back end disk problem

You can tell if you really have a back end disk problem with filemon by running these three commands:
filemon -T 20000000 -o /tmp/filemon.out -O lv,pv
sleep 120
trcstop

That should write you out a file /tmp/filemon.out that has much information in it. We extract what we are looking for with this command:
grep times /tmp/filemon.out

This should give us a bunch of read and write times. If the “avg” is greater than 20 (milliseconds), then the back end disk is not fast enough to support the load on it. Usually the solution to that situation is to add more disks and spread the load over more spindles.

Increasing I/O performance: general rules

  • Use AIO if possible (asynchronous I/O)
  • Increase queue depths if necessary (especially if you are representing large arrays of disks as a single LUN on the host)
  • Do as much work in a single disk operation as possible
  • Run as many I/O generating threads in parallel as you can. This is done on DB2 by increasing the IOSERVERS. We had good results by having one IOSERVER per spindle with a RAID10 LUN.
  • Make sure dynamic tracking is on (fscsi device attribute, dyntrk). More on this later.

Practical Steps

In reality, since there are other constraints to observe, these are usually about all you can actually do:

  • On each disk, set the queue depth to approximately: 8 * n / h

    where n = number of spindles in the biggest RAID array you are using
    where h = number of HBAs(fibre adapters) that can see the disk

    lsattr -El” and set it with “chdev-l

  • On each HBA (fibre adapter) set your SCSI queue to 2048 with “chdev -l fcsX -a num_cmd_elems=2048“. If the disks on it are in use, you will have to use this command instead “chdev -Pl fcsX -a num_cmd_elemss=2048” and then reboot for it to take effect.
  • Make sure you are using dynamic tracking (same commands as above, but with the fscsiXX devices and the dyntrk attribute- i.e. “chdev -Pl fscsi0 -a dyntrk=yes
  • If you are using AIX 5.x make sure lru_file_repage=0 in vmo. (set it with “vmo -po lru_file_repage=0“)

Use this advice at you own risk. Please be very careful when making any changes, and make sure you have backups.

References:


Have a Family Owned Business?

If you have a family owned business, or just curious about them, here is a book that my friend Brice recommended to me. Haven’t read it yet, but put it on my list:





Dynasties: Fortunes and Misfortunes of the World’s Great Family Businesses

Here is the review from Publishers Weekly:
“Beginning as a work of economics, moving through soap opera and finishing as history, this book tells the stories of 11 great family businesses in Europe, Japan and America with at least three generations of family control. Observing that the vast majority of businesses are family owned and run, historian Landes (The Wealth and Poverty of Nations) argues that dynastic businesses offer a proven route to developing emerging markets, while companies managed by unrelated professionals and funded by public investors offer mostly bad jobs and slim profit shares to local employees. Even among the largest corporations, many retain significant financial and managerial involvement by the founder’s relatives, and those that do perform better than the others. Landes’s stories emphasize emotional life within these dynasties; he includes business details and general economic history only as context for family adventures and feuds. His emphasis is on how family considerations such as authority, love, trust, envy, marriage, adoption and succession determine the growth and direction of the business. While this may seem irrational compared to entrusting strategic decisions to specialized professionals selected according to talent rather than bloodline, Landes argues that family does a better job. (Sept. 25)
Copyright © Reed Business Information, a division of Reed Elsevier Inc. All rights reserved.”


The Mythical Man Month

Managing Software Development Projects

In today’s world, more and more managers find themselves having to manage an impromptu software project. It usually starts by a need identified by a “hobbyist” colleague, who realizes that they can solve a problem by writing a simple software. Before he knows it, the manager is sucked into the world of managing software development the hard way. In professional IT shops, it is hard enough for managers to keep software projects on track. Keeping in line the requirements, the budget, and the time-line is no small feat. If on top of that, you have to add that you’re managing hobbyist software developers with no formal training or experience, that can spell disaster. While The Mythical Man-Month might not solve all your problems, it certainly will give you a new perspective making your new predicament easier to deal with. For example, when you find that your developers find themselves in a cycle of adding yet another feature and the finished product release keeps being pushed out – this is called scope creep – you will be able to rely on the guidance in The Mythical Man-Month.

This book is a required reading in reputable graduate level computer science curriculae. Here is the Amazon review of this book:


The Mythical Man-Month: Essays on Software Engineering, Anniversary Edition (2nd Edition)


The classic book on the human elements of software engineering. Software tools and development environments may have changed in the 21 years since the first edition of this book, but the peculiarly nonlinear economies of scale in collaborative work and the nature of individuals and groups has not changed an epsilon. If you write code or depend upon those who do, get this book as soon as possible — from Amazon.com Books, your library, or anyone else. You (and/or your colleagues) will be forever grateful. Very Highest Recommendation.”


Friendly Advice from my Favorite IT Shop

Ewan Reid from Utilis sent me this. Not sure where it originated. If you find the source, let me know. Definitely great advice on handling IT support:

When you call us to have your computer moved, be sure to leave it buried under half a ton of postcards, baby pictures, stuffed animals, dried flowers, bowling trophies and children’s art. We don’t have a life, and we find it deeply moving to catch a fleeting glimpse of yours.

Don’t write anything down. Ever. We can play back the error messages from our video recording.

When an IT person says s/he’s coming right over, go for coffee. That way you won’t be there when we need your password. It’s nothing for us to remember 300 login passwords.

When IT Support sends you an e-mail with high importance, delete it at once. We’re just testing.

When an IT person is eating lunch at his desk, walk right in and spill your problems right out. We don’t even like eating food, we exist only to serve.

Send urgent e-mail all in uppercase. The mail server picks it up and flags it as a rush delivery.

When we do something as a favor in our own time at our own expense, feel free to criticize us.

That’s OK, we don’t expect you to lift anything or get under your desk. Manual labour was part of our IT degree.

When the photocopier doesn’t work, call Computer Support. There’s electronics in it.

When you’re getting a NO DIAL TONE message at home, call Computer Support. We can fix your telephone line from here.

When something’s wrong with your home PC, dump it on an IT person’s chair with no name, no phone number and no description of the problem. We love a challenge.

When an IT person tells you that computer screens don’t have cartridges in them, argue. We love a good argument.

When an IT person tells you that he’ll be there shortly, reply in scathing tone of voice: “And just how many weeks do you mean by shortly?” That motivates us.

When the printer won’t print, re-send the job at least 20 times. Print jobs frequently get sucked into black holes. When the printer still won’t print after 20 tries, send the job to all 68 printers in the company. One of them is bound to work.

Don’t learn the proper name for anything technical. We know exactly what is meant by “my thingy blew up”.

When you call someone in to fix a problem – but don’t tell them about the other 10 problems until they physically arrive. That’s OK – we can clear our schedule for the rest of the day.

Don’t use on-line help. On-line help is for wimps.

When your application can’t do what you want… blame us, we write all the software that runs on your PC and can customise it on the fly. Bill Gates lets us do this.

Remember the IT guy doesn’t need to think – he has seen every problem before.

If the mouse cable keeps knocking down the framed picture of your dog, lift the computer and stuff the cable under it. Mouse cables were designed to have 20 kg of computer sitting on top of them.

If the space bar on your keyboard doesn’t work, blame it on the mail/software/network upgrade. Keyboards are actually very happy with half a pound of muffin crumbs and nail clippings in them.

When you find an IT person on the phone, sit uninvited on the corner of their desk and stare at them until they hang up.

Feel perfectly free to say things like “I don’t know nothing about that computer crap.” We don’t mind at all hearing our area of professional expertise referred to as crap.

When you need to change the toner cartridge in a printer, call IT Support. Changing a toner cartridge is an extremely complex task, and Hewlett-Packard recommends that it be performed only by a professional engineer with a Master’s degree in nuclear physics.

When you think the network/e-mail/office application is going slow, call us as we have a button to press that makes it go back to its normal speed.

When something’s the matter with your computer, ask your secretary to call the help desk. We enjoy the challenge of having to deal with a third party who doesn’t know about the problem.

The instant you call us (on our mobile) – we can see what’s happening on your screen and can solve it instantaneously.

Be aware that IT people don’t need to use the toilet. So you have a right to be upset if we don’t answer the phone.

When you receive a 30MB movie file, send it to everyone as a mail attachment. We’ve got lots of disk space on that mail server.


Tunring Filthy Water into Strile Drinking Water

From TED’s web site: :Too much of the world lacks access to clean drinking water. Engineer Michael Pritchard did something about it — inventing the portable Lifesaver filter, which can make the most revolting water drinkable in seconds. An amazing demo from TEDGlobal 2009.” Listen to the talk: http://www.ted.com/talks/michael_pritchard_invents_a_water_filter.html


IBM Expands in Analytics Market

Infoworld’s Pete Babb has an article about IBM to acquire analytics firm SPSS. Pete sees this acquisition by IBM in line with the earlier acquisition of Cognos also by IBM. He underlines that the key aspect of this acquisition is the focus on predictive analysis. IBM clearly shows a preference for solutions that work best with its core information platform DB2. Some analysts question whether IBM will fund development for compatibility on competing information platforms such as Oracle and the emerging Open Source options. According to SAS – one of the main competitors of SPSS in that field – they may have an opportunity to increase they market share over SPSS if IBM will focus on DB2 at the expense of other information platforms.


Installing Product License for DB2

In a previous blog post I described how to upgrade to DB2 9.7. However, once the upgrade is complete, the old 9.5 license becomes invalid and a new license needs to be loaded. Preferable before you fire up the new version. Loading the new license is quite simple:

  • Download the license file from IBM (it is in your downloads or it can be on a CD that cam with your product)
  • Load the license with
    db2licm -a path/to/license/file
    the db2licm executable is typically in the adm folder

References:


Creating Delete Triggers in PostgreSQL

One of the common data integrity issues that can happen in a database is the unintended deletion of a row. Here is how to create a DELETE trigger in PostgreSQL.

The example code below assumes you have a “customer_cus” table and a “customer_archive_cua” table with at least two fields. It is a good idea to add a timestamp field to the archive table with DEFAULT now(), so that the date of deletion is captured.

CREATE FUNCTION archive_customer() RETURNS TRIGGER AS '
BEGIN
INSERT INTO customer_archive_cua (name_cua, address_cua)
VALUES (OLD.name_cus,OLD.address_cus);
RETURN NULL;
END;
' LANGUAGE 'plpgsql';
CREATE TRIGGER customer_archive_on_delete AFTER DELETE ON customer_cus
FOR EACH ROW EXECUTE PROCEDURE archive_customer();

References