Testing – Best Kept Secret in Software Development

The majority of projects in IT are over budget and miss their deadline.  We often notice that IT staff works hard just to stay in one place.  While there are many causes, one of the key factors to mitigate the problem is testing.  In this article, we will define show what happens if you:

  • let testing become an afterthought.
  • let testing to be reckoned with after the software exists.
  • fail to consider all kinds of testing: unit, system, performance, and acceptance testing.

When these happen, the benefit of testing will be very limited: thumbs up or down on software, the final mark of quality, or the lack thereof.  What are the opportunities missed with this view?

Requirements testingsoftware testing

Hardly anyone disputes that a lack of clear, well understood and documented requirements is a land mine likely to blow in the development cycle, often causing overruns in both timelines and budgets.  Yet a lot of companies miss the low hanging fruit in this phase: creating test cases, where each test case corresponds to each functional requirement.   (Think of specifying technical requirements and performance requirements in this phase to improve effectiveness.)  More often than not we find requirement “bugs” using this simple method. At this stage, we can easily clarify and fix them.  Cost of defect fixing is exponential – 1x in requirements, 10x in testing after coding and 100x in production.  This is often referred to as the 1:10:100 rule.*

If we find a bug when the software is live, it is likely tarnish the reputation of the software supplier or service company. We have seen when a bug can even sink it. The example of Sybase in the area of RDBMS demonstrates this best.  (Who remembers today that they used to be Oracle’s strong competitor?)  Yet how many times does a team rush to design? How often do we see people making assumptions and finding out these were incorrect in the latter stages of development?

Frequency of testing and test automation

It’s more a rule than an exception that monolithic projects with a massive delivery at the end – often nicknamed “the miracle happens here” – don’t work.  We believe that small deliveries have the advantage of realizing business advantage early antesting between releasesd enabling adjustments.  Whether you practice true ‘agile’ methodology or phased development, testing is an integral part of this cycle.  When we optimize testing using automation tools, it saves both elapsed time and costs.  In small phases, this becomes a necessity, as developers can leave testers in the dust when doing sprints with manual testing, with all of the negative ramifications.  

When we developed automated testing, we realized that most of the tools require scripting and in turn need a qualified scriptwriter. As it takes time to become a scriptwriter, we noticed in many cases teams often end up using developers to write scripts. As a result, we found that scriptwriters end up creating bottlenecks. That would increase costs and create delays.  The solution we found was to use tools which allow test creation through GUIs so that anyone can create such tests.  

Summary

Finding bugs early is cheaper and saves time.  It is key to maximize ‘test coverage’ defined as the percentage of the code covered by test cases.  A simple matrix illustrates the challenge:

Bugs Known Unknown
Major X
Minor

It is the major unknown bugs which present the biggest challenge – we don’t know what we don’t know.  How do we minimize this quadrant?  The higher the test coverage the lower the likelihood of unknown bugs.  Automating testing through tools which provide agility in the creation of test cases and updates in conjunction with doing test cases parallel with requirements can significantly change the probability of success of your project while decreasing its cost.  Why not do it?

At NewPush we provide Testing as a Service to help start testing. Our service offers minimum difficulty while reaping all the benefits.  There is no prior knowledge of testing needed. We create automated test cases which can be run at any time.  We also provide project management consultancy and we can create test cases upfront during the requirements phase.

*Ralph Young: Effective Requirements Practices: Defining the Real Customer needs.


Tips and best practices for migrating a legacy website into Drupal 7 with the Migrate API

drupal_articleLast year, we have migrated a site from a legacy CMS with a lot of custom code into Drupal 7. With the Migrate module (which is now in Drupal 8 core by the way), it was not really hard to do, even though the source database was not really well-designed. The site had around 10k users and 100k entries which ended up being nodes in Drupal, with around 700 videos to migrate. While this is still not a huge migration by any means, it was big enough to make us think about the best practices for Drupal migrations.

We have collected some tips and tricks if you need to do anything similar:

Building the right environment for the migration work

If you would like to work on a migration efficiently, you will need to plan a bit. There are some steps which can save you a lot of time down the road:

  • As always, a good IDE can help a lot. At NewPush, we are using PhpStorm – in that, keeping both the migration code, the source and the target databases opened is straightforward. Also, it is really easy to filter the database table displays looking for a given value which comes really handy. Anyway, any tool that you can use is fine, as long as you are able to quickly check the difference between the source and target versions of the data, which is essential.
  • I know this can be a bit of a pain sometimes, but still: try to ensure that the site can be built from scratch if necessary. This is where techniques like Features step in. I will not explain this in detail since it is outside the scope of the article. Just make sure you ask yourself the question: “what if I really mess up the configuration of this site?” (In a real world scenario you will often need to adjust things like content types, field settings etc. etc. and you will probably need a method for keeping the final configuration in a safe place.)
  • Before getting started, try to figure out a good way for validating the migration. This is kind of easy if you have 20-30 items to move over, but when you need to deal with 50k+ nodes, it is not going to be that trivial – especially if the source data is not very clean.

How to work with your migrations

The next thing to work on is optimizing your day-to-day work. Making sure that you can perform the basic tasks fast is essential, and you will need to figure out the best methods for testing and avoiding errors as well.

  • Use Drush as much as possible. It is faster and less error-prone than UI. There are a lot of handy parameters for migration work. Talking about the parameters, the –feedback and the –limit switches are really handy for quick testing. With the –idlist parameter, you can exactly specify what to import, which is great for checking edge cases.
  • Try to run the full migration from time to time. This is usually not very convenient in a development environment, so having access to another box where you can leave a migration running for hours can make things quite a bit easier.
  • Don’t forget to roll back a migration before modifying the migration code – it is better to avoid database inconsistency issues.

Migration coding tips

Talking about coding the migration itself, there are several things to consider, most of them are pretty basic coding best practices:

  • Try to extract the common functionality into a parent class. You will probably need some cleanup/convert routines; this is the best place for them.
  • Try to document your assumptions. Especially when dealing with not-so-clean data this can be really useful. (Will you remember why the row with ID 2182763 should be excluded from a given migration? I will not, so I always try to add a bunch of comments whenever I find such an edge case.)
  • Use the dd() function provided by Devel – this can make things easier to follow.
  • You will most likely run into the “Class MigrateSomething no longer exists” error at some time. Give it a drush migrate-deregister –orphans and it will go away.

How to optimize the performance of your migration code

It is no surprise that running a full migration often takes a very long time.

  • Other modules can greatly affect the speed of migration/rollback. Try to migrate with only a minimal set of modules to speed things up. This is the easiest way to get some extra speed. (And also if you are on a dev machine, make sure that Xdebug is not active. That is a “great” way to make things much slower.)
  • Using the migrate_instrument_*() functionality, you can figure out the bottlenecks in your migration.
  • In hook_migrate_api(), you can temporarily disable some hooks to speed up the migration.

I hope you have found some useful tips in this article. If you have a question, a tip to add, or perhaps an interesting story about a migration that you have done, just share them in the comments, we are all ears!