How to Improve Continuous Integration

CI (Continuous integration) is common practice in agile software development (modern software engineering). But this is an expensive solution in terms of the time to receive feedback and requiring high computational resources.

This cost should not be underestimated; our personal correspondence with a Google manager in charge of their CI system TAP reveals that TAP costs millions of dollars just for the computation.

There are many best practices to introduce the more efficient CI process like

1. Maintain a code repository
2. Automate your build
3. Make your build self-testing
4. Daily commits to the baseline by everyone on the team
5. Every commit (to the baseline) should be built
6. Keep your builds fast
7. Clone the production environment and test there
8. Make it easy to get the latest deliverables
9. Everyone on the team can see the results of your latest build
10. Automate build deployment

But…this is pretty common sense even though many companies are trying to achieve them. But there was a more fundamental try to improve CI by Xianhoa Jin and Francisco Servant.

The main focuses of the experiment are 1) Time-to-feedback-reduction and 2) Computational-cost-reduction by having 1) (build/test) selection techniques and 2) (build/test) prioritization techniques. Of course, there are trade-offs between the level of reduction and missed failure observation, which means the case needs to be built/tested (but skipped as it’s selectively done) as it is a failure case.

This is the case not only in the giant IT companies like Google and Microservice but also the small-size company as they have relatively fewer resources for CI.

With the thesis, I can conclude the selective test/build the code commits will eventually save a lot of cost in computational cost but there will be the risk for the missed failure case. And prioritization of the test/build is great unless the code is independent and there is great logic to know which one is a higher priority (same as above, how to select the code commit to being tested/built or not).

Reference

• Usage, Costs, and Benefits of Continuous Integration in Open-Source Projects (ASE 2016: Proceedings of the 31st IEEE/ACM International Conference on Automated Software EngineeringAugust 2016)
• What helped, and what did not? An Evaluation of the Strategies to Improve Continuous Integration (arXiv:2102.06666 [cs.SE])
• https://www.cloudbees.com/continuous-delivery/continuous-integration