Introduction to Apache Skywalking

Appache Skywalking is an Application Performance Monitoring tool for distributed systems, especially designed for microservices, cloud native and container-based (Kubernetes) architectures.

aiTest uses Apache Skywalking tool to monitor the Application’s performance when URL test, Performance test or Multi Browser Test is runned in aiTest for the Application. Monitoring tests using Apache SkyWalking brings significant business value, including enhanced test visibility, proactive issue detection, performance optimization, scalability assessment, service dependencies analysis, root cause analysis, and continuous improvement.

Data flow

  
    Skywalking-agent(setuped in Application non-intrusively) -> 
    skywalking OAP server -> skywalking UI
  

Features

1. Topology : Topology map to understand a distributed system, it visualizes the entire complex system in an easy-to-read layout.
2. Metrics : service, service instance, endpoint metric analysis.
   • SYSTEM : cpu, memory, network, process
   • Service Avg Response Time (ms)
   • Service Apdex(Application Performance Index) is a numerical measure of user satisfaction with the performance (It is the ratio of satisfactory response times and unsatisfactory response times to total response times.)
   • Service Load (calls / min)
   • Success rate (%)
   • Service Instances Load (calls / min)
   • Slow service Instance (ms)
   • Service Instance Success Rate (%)
   • For Current service : endpoint load, success rate, Slow Endpoints in Current Service (ms)
3. Traces : Tracing shows you event behavior over time, so that you can track a request from start to finish and identify system defects and errors.
4. Logs : detailed Logs provide a record of events or error messages.

Application Integration

For python agent

1. In DockerFile RUN pip install apache-skywalking
   RUN pip install apache-skywalking
2. In DockerFile CMD or ENTRYPOINT
   *add sw-python run before the command to start application.
   example :python manage.py makemigrations && python manage.py migrate && sw-python run python manage.py runserver 0.0.0.0:8000
3. Add environment varibales
   SW_AGENT_NAME: "Service name"
SW_AGENT_COLLECTOR_BACKEND_SERVICES: ExternalIP:11800

   For Java Agent

4. In DockerFile RUN this command
   RUN wget https://dlcdn.apache.org/skywalking/java-agent/8.13.0/apache-skywalking-java-agent-8.13.0.tgz && tar zxvpf apache-skywalking-java-agent-8.13.0.tgz && rm apache-skywalking-java-agent-8.13.0.tgz
5. In Entry point of Dockerfile add JAVA_OPTIONS
   ex: ENTRYPOINT ["java","-cp","app:app/lib/*","-javaagent:skywalking-agent/skywalking-agent.jar","com.example.polls.PollsApplication"]
6. Add environment varibales
   SW_AGENT_NAME: 'Service name'
SW_AGENT_COLLECTOR_BACKEND_SERVICES: ExternalIP:11800

Database(MYSQL) Monitoring

How the telemetry data is collected and sent

1. mysqld_exporter collect metrics data from MySQL/MariaDB.
2. OpenTelemetry Collector fetches metrics from mysqld_exporter via Prometheus Receiver and
3. pushes metrics to SkyWalking OAP Server via the OpenCensus gRPC Exporter or OpenTelemetry gRPC exporter.

Steps to setup for MYSQL database Monitoring

1) setup mysql_exporter in the docker-compose.yaml

  
    mysql-service:
        image: prom/mysqld-exporter:v0.14.0
        ports:
          - 9104
        environment:
          - DATA_SOURCE_NAME=${USER}:${PASSWORD}@(${DB_SERVICE_NAME_IN_DOCKER-COMPOSE}:${DB_PORT})/
        networks:
          - ${NETWORK}
        depends_on:
          - ${DB_SERVICE_NAME_IN_DOCKER-COMPOSE}
  

2) Setup Otel Collector in docker-compose.yaml which will have config for prometheus receiver, processors: batch, exporters : otlp

  
    otel-collector:
        image: otel/opentelemetry-collector:0.67.0
        networks:
          - ${NETWORK}
        command: [ "--config=/etc/otel-collector-config.yaml" ]
        volumes:
          - ${LOCATION_OF otel-collector-config.yaml}:/etc/otel-collector-config.yaml
        expose:
          - 55678
        depends_on:
          - ${DB_SERVICE_NAME_IN_DOCKER-COMPOSE}
  

3) Create otel-collector-config.yaml file for prometheus receiver, processors: batch, exporters : otlp

  
    receivers:
      prometheus:
        config:
        scrape_configs:
          - job_name: 'mysql-monitoring'
            scrape_interval: 10s
            static_configs:
              - targets: ['mysql-service:9104']

    processors:
      batch:

    exporters:
      otlp:
        endpoint: ${SKYWALKING_SERVER_EXTERNAL_IP}:11800
        tls:
          insecure: true

    service:
      pipelines:
        metrics:
          receivers:
            - prometheus
          processors:
            - batch
          exporters:
            - otlp
  

Questions answered

• How aiTest works?
• What I do after Sign-Up?