Columns

Will BI projects go offshore?

• By Wayne W. Eckerson
• May 1, 2004

The loss of U.S. information technology jobs to overseas consulting firms is alarming. Data-quest estimates that 140,000 IT jobs were outsourced to India last year. Some experts claim this "trickle" will grow into a flood as software firms and corporate IT departments rush to benefit from the "Delhi discount." To date, U.S. programmers have been hit the hardest. A few years ago, programmers were the hotshots of the dot.com era, often commanding six-figure salaries and options. But in the past two years, salaries for application developers have dropped 17.5%, while salaries for database engineers have gone down by 14.7%.

U.S. programmers are now competing against highly skilled, highly educated, English-speaking programmers in India and elsewhere who are happy to work for one-eighth the salary of their U.S. counterparts.

When the exported jobs were not glamorous -- testing, code migrations and maintenance -- no one seemed particularly threatened. But now overseas outsourcing firms, such as Tata Group, WiPro Technologies and Satyam Computer Services, are everywhere and bidding on all kinds of projects -- from Web development to data warehousing and business intelligence. It seems as if any technical job that does not require physical proximity is a candidate for outsourcing. Everyone feels a tad threatened for their own jobs, as well as for the long-term prospects of their children.

Can we stem the tide?
Before we urge our government leaders to pass protectionist policies, we should remember that unemployment in the software industry today is at 7%, according to a recent BusinessWeek article ("Software: Will outsourcing hurt America's supremacy?," March 1, 2004). Also, the outsourcing of IT jobs is not a new phenomenon. U.S. companies have outsourced software development and other jobs for years to Canada and Ireland, where lower wages have translated into huge project cost savings. Finally, as the theory goes, outsourcing helps companies to become more cost-efficient, productive and competitive, which in turn spawns new products, projects and -- finally -- jobs.

Although it is tempting to paint apocalyptic scenes based on the current exodus of well-paying IT jobs overseas, we must remember that we've been through this before. Every time, American innovation has responded to the challenge and created new opportunities for its businesses and workers. We're already seeing ways in which IT workers are developing new techniques to increase their productivity and competitiveness. And as our economy rebounds, we'll see U.S.-based firms unveil new products, technologies and visions for the future that will more than recoup our current losses.

Does offshoring work in BI?
Although offshoring has racked up success in traditional IT development, it's unclear whether this success transfers to the data warehousing and business intelligence (DW/BI) arenas where the key to success is having the technical team work hand-in-hand with business users. Many data warehousing managers tell stories in which offshore teams cost client firms more money than they were saving due to miscommunication or lack of knowledge about data warehousing.

In addition, some argue that firms would be silly to outsource a valuable corporate asset -- integrated data -- because of competitive concerns and privacy laws. Outsourcers may commingle data from multiple organizations within the same data center, the argument goes, and thus not be able to protect that data from falling into the wrong hands.

Making it work
Offshore consultants admit that they have made mistakes in the data warehousing domain, but they've learned from them. Many have learned data warehousing "on the job" -- often at the client's expense -- but now have the domain expertise to deliver quality results. They have also implemented processes and controls to ensure better, more efficient communications between onshore and offshore teams.

For instance, offshore firms realize that certain positions must remain on site: analysis, design and end-user support, as well as one or more project managers. In addition, firms have learned that they need to keep an offshore development team working during U.S. hours to resolve immediate issues and concerns.

Of course, these steps often increase the cost of having the offshore team doing the work in the first place. The hiring firm must discover the break-even point when the cost savings from offshoring are not worth the hassle of maintaining a development team halfway around the world.

Based on my research, here are a few keys to making the offshore model work in data warehousing:

1. Use a trusted partner. The most successful offshore projects are done by consulting firms that have proven themselves to the client company over the course of many projects and years.

2. Go slowly. Don't outsource the entire data warehousing project. Outsource easy items and tasks, such as ETL coding or unit testing, so that you can evaluate the skills of the offshore team and see how the relationship works. This allows you to refine communication processes in a less stressful environment.

3. Assess experience and stability. Understand how much data warehousing experience the members of your offshore partner have. Also find out how stable their team is. You don't want key players to turn over during your project, especially on-site project managers who can be easily lured away by the competition or other U.S. jobs.

4. Assess security. If the security and confidentiality of your data is a serious concern, avoid doing business with offshore firms that work with your competitors or in your industry. Also find out what controls they apply to workers who access your data and how they partition their data centers to avoid commingling data from multiple firms.

5. Evaluate processes. Make sure the offshore team has put in place processes and controls to ensure effective communications. This includes using detailed requirements and specification templates and establishing clear channels of communication between on-site and offsite project managers and developers.

6. Ensure technology transfer. Make sure you don't outsource your ability to run and maintain the system and educate users once the project is over. Make sure that the offshore firm transfers their knowledge to your team members.

Execution is the key
As in any data warehousing project, the key is execution. Offshore consulting firms will eventually figure out how to deliver data warehousing projects efficiently and effectively. In fact, their real advantage may not be their cost structures, but their recognition that the key to their success is ensuring effective communications. Offshore teams may end up overcompensating for this intrinsic deficiency in their model and surpass home-based development efforts.

I've seen many U.S.-based data warehousing projects fail because the business and technical teams couldn't or wouldn't communicate. It didn't matter that they were located on the same floor in the same building. They may as well have been working on different continents and speaking different languages. Just because developers are born and bred in the U.S. doesn't guarantee that your data warehouse will be a success.

In the end, the key to the success of any data warehousing project is execution. The nationality of the person who runs the project doesn't matter, nor does where the work occurs. If the project manager is highly competent, the developers understand the business and have strong technical skills, and there is tight alignment between the business and technical teams, then a project will invariably succeed.