Android Developer Challenge

As many of you already know, Google has launched the Android Developer Challenge. The challenge will give away $10 million dollars in awards for "great" mobile apps built on Google's Android platform. Being the past founder of a mobile applications company, I could not resist. The ideas started flowing and after several brainstorming sessions with a colleague of mine, we came up with our "great" mobile app.

The colleague of mine is Michael Cook, past founder of CookWare, Inc. At this point, all we have are a few requirements documents, a slide show, and a basic Android "hello world, testapp" application. I never expect much when entering challenges like this, but who knows? Wish us luck!

Chasing a Constantly Changing API

I was once waist deep into a development effort when a partnering development team informed us that they where changing their API again. The component we were developing depended on their components for data and capabilities. We had spent time interfacing with their component. We had been careful enough to isolate the interface to their API by creating a façade. This way, the two components were decoupled as much as possible and our component was somewhat isolated from future changes to the partners API.

Our partner was already changing their interface. Not a problem, right? The façade is the only subcomponent affected by this change. An engineer spends a few days reworking it and we move on. It’s that easy right? Well, no.

When this happens you need to make sure you find out why their interface is changing. Ask your partner why this change has been made and why this change was not thought of originally. If they do not have solid reasons for this, then be assured that their API will soon change again and you will find your team spending another couple of days updating the façade. I have seen this situation play out many times. Each time the partner swears that "this is the final change to the API" only to have it change again a week later.

Yes, the façade acts as a buffer to new changes but changing the façade still requires work. And no façade is completely perfect. Some changes might still bleed into other subcomponents causing many other needed changes. In fact, the more you need to update the façade, the greater the chance that the changes will effect all the subcomponents who utilize the façade.

In many situations you have no control over your partner's development practices. It is important to first identify when you may be dealing with an unstable API. Once you have identified the API as unstable, you have a few options other than using up all your resources chasing it.

1. Hold off on making the changes. Wait a few weeks before updating your façade to their latest API. You might also be able to wait till the end of the development effort to make the update. This way, you will skip a few interim API changes. I do have a word of caution on this. Make sure you’re not ignoring their changes; only skip the updates to your code. You want to make sure their changes do not drastically change or remove the needed data or functionalities you require from them. If you wait till the end of development, it might be too late to address the problem.



2. Send you partner your façade. Many partners will pushback on this. They might see this as more work for them, but it might actually save time for both of you. They have a better understanding of the new API and it will be a quicker change for them to make. They already know what changed in the API and why, thus cutting down on cross team meetings and emails.

Make sure you identify their unstable API as a risk to the project as early as possible. Assess your options; determine if your schedule, budget, or scope for the project is at risk. Chasing a changing API never seems like a big deal to the project until it is too late. If it is not identified and dealt with early then it will put the whole project at risk for failure.

Engineers and Marketing Myopia

For those who are new to SheeleyTech, I am a software engineer at heart but more often I “act” as a project manager, product manager, and systems engineer. Because of this, I tend to find myself in a conflict of interest that, if I’m not careful, results in marketing myopia.


“Heads Down” Versus “Heads Up” Roles

The engineer’s role
Engineers love to learn and apply their newly acquired skills. They enjoy building products by using their knowledge, skills, and past experiences to design and create solutions. A large part of this enjoyment is the challenge of creating the product. Sometimes it is the ability to utilize a technology that the engineer has never used before. As a software engineer I can look back at the software products I enjoyed developing the most. There is the first time I used CORBA, or the first time I used RMI, or learning about and developing web services was a great experience. Keeping up with the latest technologies, gaining new skills, and creating products with new tools requires the engineer to focus on the development of the product. I refer to this as a “heads down” responsibility. The idea being that the engineer’s head is facing down at the computer screen writing code or reading about new technologies.

The product manager’s role
When you are in the role of product manager, systems engineer or even the project manager you need to be focused on the customers’ problems. What needs do your customers truly require? Your job is to make sure you are building a product that solves their problems. You need to be what I call “heads up” – researching competing products, learning about the customer, and gaining feedback from the customers. This process does not stop once the development effort of the project begins. Challenges and issues will come up during development. Decisions will be made halfway or ever later into the development of a product that will have a huge impact on the final product. These challenges will come from the development effort (technical issues) as well as from the customers (market issues). Both of these types of challenges need to be addressed with the interests of the customers first.


The Conflict of Interest

When the same person developing the product is the same person who is responsible for making sure the product will meet the customers’ needs, there is a natural conflict of interest. I speak from experience.

Once the development effort starts, the engineer begins to fall in love with his or her product. There has been long hours invested into their development. Thus the engineer has been “heads down” and the focus has been on the product. When technical issues come up, the first reaction is to find the best technical solution to the problem. This is how the engineer wants to solve the problem. Because the product manager and the engineer are the same person, the interest of the customer can get lost. For example, a drill bit engineer (if there is such a thing) will focus on the drill bit. The engineer begins to think that the drill bit is the focus and that the customer needs a really good drill bit. This is wrong. What the customer truly needs is a hole, not a drill bit. Issues need to be addressed by thinking about the need of a hole, not how to make the drill bit better.

If you share roles as an engineer and as a product manager, you need to spend equal time “heads up” as well as “heads down”. If you started your career as an engineer you need to fight your instincts to focus more on the product than the customer. Before dealing with any technical issue, take a step back and think only about the customers’ needs. Review any relative requirements created for the product. Revisit use cases and all the materials you created before you put your head down into the code. Seek out subject matter experts or even contact the customer to get their perspective on the problem. This way, in the end, you will have developed a product the customer needs rather than just a product you enjoyed developing.

Negotiated Schedules Affect Quality and Deadlines

A while back I was at a meeting with a group of product managers from local software companies. The topic was the process of prioritizing requirements. When the estimated time of development came up, one product manager made the following comment:

“Estimates from engineers are always overestimated. They would want you to think all features would take about three years to develop. You need to negotiate the schedule.” …or something like that.

I could not believe what I was hearing. From my experience, engineers are usually over optimistic with their initial estimates. Only after thoroughly examining each essential step will an engineer increase their approximation to a more reasonable estimate. The fact of the mater is, software development efforts are notorious for delays and schedule overruns. This negotiation process is likely a common cause.

My guess is that this product manager probably feels a sense of accomplishment when the engineering team finally agrees to shorten their schedule. He probably then gets frustrated later when new releases are delayed. Product managers, project managers, program managers, and anyone else who has a say in persuading schedules must realize negotiating a development schedule does not mean the project will finish any earlier. The estimated delivery date is just that, an estimation. The actual delivery date will not be effected by the new estimations. It will not cause engineers to work any faster, it will not reduce the amount of work needed to be completed, and it definitely will not increase the number of hours in the day. The only thing a negotiated schedule will affect is a low quality product or a delayed delivery, and that is not something any manager should feel a sense of accomplishment about.

Testing Early and Often Can be a Wasted Effort

We have all heard the saying, "test early, test often" but in many software development efforts this can be a very costly mistake. It is imperative for software project managers to understand the full added cost each type of testing will have on the final delivery date. In many cases, it is the timing of when to write these tests that matter the most.

There are three levels of testing, unit testing, integration testing, and system testing. Integration testing and system testing are not as flexible, so I will not discuss these two efforts in this post. Unit testing does have some flexibility.

Unit testing is a required means to assure that the smallest possible elements have been tested. Unit testing is a required process in testing a system, no question. But the timing of these tests and the level of detailed required need to be considered. Writing highly detailed unit tests early in a project adds cost to the current development phase but, more importantly, adds cost to future phases as well. Maintaining these tests can be extremely time-consuming. In iterative process models, units are rewritten and redesigned over and over again. Each change to unit requires changes to the corresponding unit test. The more detailed the unit test the more time and effort required to maintain and update.

Project management needs to be aware of the added costs and implications to the project’s schedule. The nature of the project, the type of development process, and the size of the system being developed all need to be considered in deciding the unit testing level of detail and what phase or iteration these levels should be enforced.

Unit Testing Levels of Detail
First let’s clearly define each levels of testing. They are numbered and described below from the least complex level to the most complex.

1. Algorithm Accuracy testing
- Testing the accuracy of an algorithm. In object oriented languages, this is
testing the return of a method based on a few basic inputs.

2. Full statement testing
- Making sure the unit tests execute every line of code

3. Full branch testing
- All decision points (i.e. if statements) are executed for all outcomes

4. Every loop tested at 0,1 and many
- Testing each loop (i.e while and for loops) when the loop is executed more
than once at a time (many), only once (1), and when the condition of the loop
keeps the loop from executing its block of code at all (0)

5. Boundary value tests
- Testing boundary values for boundary condition statements (i.e. if(size > 0
&& size <= 10) Boundary value tests would test this statement when size equals -1, 0, 10, and 11 )

6. Full predicate testing
- Similar to level 2, Full branch testing, adding the requirement for each
condition for each predicate must be tested. (i.e if(boolOne == 1 &&
boolTwo == 2) Full predicate testing would tests all combinations when boolOne
equals 1 and does not equal 1 and boolTwo equals 2 and does not equal 2)

7. Path testing
- This level of testing implies all the previous levels but each branch, loop,
boundary, and predicate test must be done independently of the others.

Each level of detail adds more benefit to the reliability, accuracy, and even the complexity of the system. There is no question about this. Determining what level of detail is required by the stakeholders needs to be done regardless of the development process chosen, but understanding the cost of each level is important in estimating cost and schedule.

Software Development Processes
We’ll now describe the basic types of development processes. Depending on the project’s process, the costs of testing differ greatly.

Waterfall process – a sequential effort of phases (requirements analysis, design, implementation, testing (validation), integration, and maintenance). Each phase occurs only once. Traditionally, testing occurs once the implementation phase finishes but many times unit testing occurs during implementation. Either way, these efforts only occur once.

Iterative process – a non-sequential effort where phases can be revisited after there initial completion. A common iterative process, Agile process, comprises of short iterations. Each iteration visits or revisits each software phase. Think of an iteration as a small waterfall. Iterative processes reduce the level of risk to a project by giving the stakeholders of the project small releases of the product at the end of each iteration. The stakeholders are able to evaluate these releases and add feedback, suggest changes, and foster new ideas for the product. Stakeholders can better evaluate whether to continue on with the project or not. If continued, the project can redesign, and recode the product using the stakeholders’ feedback. This method ensures that the project and the stakeholders are communicating towards build a better product throughout the effort. This is contrary to the waterfall processes that only allow stakeholders to evaluate working examples of the final product at the very end of the effort.

If you are using the waterfall development process, once the implementation phase is done, you will not revisit this phase. Therefore, determining the cost attributed to each level of detail is simple. If an iterative process in being used, each level of detail will increase by a multiple of the iterations in the process. In many iterative development efforts, the code written in the first and even the second iterations will greatly change once the stakeholders give new feedback and make new requests. Writing highly detailed unit tests for these iterations will most likely be a wasted effort. Proceeding iterations will also have the added burden of editing and changing these tests.

Making sure these tests are written efficiently can cut down on the complexity and the level of effort needed to maintain these tests. Tools such as the Spring Framework can help to cut down drastically on the complexity of the setup code needed for each unit test and should always be used on medium and large efforts.

Levels of details such as levels 4, 5 and 6 should be post postponed until the system and software requirements have been mainly worked out. These levels add a large amount of extra code to the test suite. Not until the stakeholders seem to settle on the majority of requirements and details of the system should these levels of detail be added to the test suite. Although these tests greatly increase the quality of the software, their cost to write and maintain surpass their benefits to the early iterations.

Level 7, path testing, is a level of detail that adds great quality to a system, but it is extremely expensive. A testable section of code’s complexity can great affect the cost to create and maintain tests at this level. Some argue that requiring engineers to write full path tests influence engineers to keep the complexity at a minimum. I agree with this point, but in the early iterations of a development effort code complexity is irrelevant. These iterations are done to prove a concept to the stakeholders and to iron out the integration effort problems and system level issues. The last iterations before a system is released should be saved for this level of detail. The added costs in adding these tests to the system will be outweighed by the time and effort required to maintain these tests.

Conclusion
Every software project is unique and it is up to the management to understand the costs involved in each decision they make. Test adds quality but also adds cost. Test wisely adds quality but can reduce cost. As a project manager, make sure you understand the complexity your decisions add to future phases and iterations of the project. More detailed testing can be postponed until later interactions where there is less future maintenance needed to update the tests.