/
Project Setup

Project Setup

Owned by Daniel Guerrero
Jun 14, 2024
3 min read

To setup a project there are two main steps:

  1. Create project in sonarqube

  2. Add support in project files

Create project in sonarqube

  1. In projects page click on button Create Project select Local Project

Step 1

  1. Set Project display name a meaningful name

  2. Project key must be the repository id

  3. Main branch name should be develop

image-20240517-214311.png

Step 2

  1. Choose the baseline for new code for this project
    Select Use the global setting

     

On Analysis Method page there is no need to do anything for now so can be skipped

Add support in project files

Add sonar-project.properties file

The sonar-project.properties must be created on top folder with following:

sonar.host.url=https://sonarqube.touchcastmaas.dev/ sonar.token=ANALYSIS_TOKEN sonar.projectKey=PROJECT_KEY sonar.python.version=3.11 sonar.sources=/app/src sonar.tests=/app/tests sonar.python.coverage.reportPaths=/app/tests/coverage.xml sonar.python.xunit.reportPath=/app/tests/unit-test.xml

This project is for a python project note that the /app prefix will be hardcoded and there is no need to have the same folders in the codebase, but this is the usual naming in projects

Change unit testing files

The unit testing files should include two things: unit testing reports and coverage report the paths must match the sonar-project.properties so the base file for tests is:

import logging import unittest import xmlrunner import os import sys import coverage from typing import IO, Tuple logging.basicConfig( level=logging.INFO, format="[%(asctime)s %(levelname)s] (%(threadName)s::%(funcName)s) %(message)s", ) logger = logging.getLogger(__name__) def coverage_start() -> Tuple[coverage.Coverage, IO]: output_fp = None # default path if run through venv src_path = "src" unitest_prefix = os.environ.get("UNITTEST_PREFIX") if os.path.exists(unitest_prefix): # this is only when running through docker basepath = unitest_prefix else: basepath = os.path.realpath(basepath + f"/../") coverage_basepath_code = os.path.join( unitest_prefix, src_path, ) sys.path.append(coverage_basepath_code) # extend here all the paths you want coverage source_paths = [coverage_basepath_code] coverage_instance = coverage.Coverage( source=source_paths, ) coverage_instance.start() if os.environ.get("XMLRUNNER_OUTPUT_FILE"): output_fp = open( os.environ.get("XMLRUNNER_OUTPUT_FILE"), 'wb', ) return (coverage_instance, output_fp) def coverage_stop(coverage_instance: coverage.Coverage, output_fp: IO): coverage_instance.stop() if output_fp: output_fp.close() if os.environ.get("COVERAGE_OUTPUT_FILE"): coverage_instance.xml_report(outfile=os.environ.get("COVERAGE_OUTPUT_FILE")) if __name__ == "__main__": basepath = os.path.dirname(os.path.realpath(__file__)) basepath_cases = os.path.realpath(basepath + "/cases") sys.path.append(os.path.realpath(basepath + "/lib")) runner_extra_params = { "verbosity": 2, "buffer": True, } coverage_instance, output_fp = coverage_start() loader = unittest.TestLoader() tests = loader.discover(pattern="*_test.py", start_dir=basepath_cases) if output_fp: runner = xmlrunner.XMLTestRunner( output=output_fp, **runner_extra_params, ) else: runner = unittest.runner.TextTestRunner( **runner_extra_params, ) response = runner.run(tests) coverage_stop(coverage_instance, output_fp) sys.exit(0 if response.wasSuccessful() else 1)

This file must be in tests/ folder

If source path is different from src it must be changed in line 20 (src_path variable)

Note that this will include all the paths under src/ folder to be covered, if there are need to remove some folders (like configuration files, etc) it can be restricted to the pats in line 32

 

The requirements-dev.txt for this are:

coverage~=7.4.3 unittest-xml-reporting~=3.2.0

 

You will need to ignore the output files for sonarqube (.gitignore)

 

Makefile

The unit_test step needs to be something like this:

The core part here is to pass the following:

The environment variables are needed by the unit test script, and the volume is needed to force being under /app folder (because sonarqube needs absolute paths)

 

The second change is to add a sonarqube_scanner step

This will generate and upload the report to sonarqube

 

Change the test script to add a step to convert relative paths into absolute

Add the following at the end of tests/run_tests.sh

 

Update Jenkinsfile

For now the scan is only made on the develop branch, so it needs the following in Jenkinsfile to achieve this

 

After deploy / publish call