Categories
Uncategorized

DevOps platforms

DevOps is defined (from Wikipedia): DevOps (Development and Operations combination of words) is an emphasis on “software developer (Dev)” culture of communication and cooperation between the “IT operation and maintenance technicians (Ops)”, movement or practices . Through Automation “software delivery” and “schema changes” in the process, to make the build, test, distribute software to be more fast, frequent and reliable.

Technology companies are currently hundreds of people about it, but the whole technology stack is still relatively backward, especially DevOps, this one container, you need to open up across the board, was also largely responsible for DevOps come in this piece of work, it should be said is not how to do well, which is also quite a few detours, below the main pit it is stepped on their own.

A free-style software project

The main building is a free-style software project based jenkins inside, then the reference is Ali codepipeline, is one of jenkins package, including the creation of job, build up today and get building progress so encapsulated and will be required to keep things library, did not think the code code, a bunch of pits, such as: 1. click on the continuous build immediately, jenkins is out of order returned (distributed lock resolved) 2. cross-domain calls, csrf, the better, but jenkins easy to engage unable to log (note configuration, specifically click here) 3. create a job when only supports xml format, but also convert it, the super pit (xstream forced conversion) 4.docker building when the need to mount sink docker host (I thought of using the remote, but the efficiency is not high) job consistency 5. database and jenkins, mission creation fails, bulk delete too slow (currently thought better how to solve) 6. due to the use of the database, you need to detecting whether the job is built, in order to customize the parameters, we wrote a notice from the plug, will build a state return to kafka, then During message processing management platform.

He completed more than things, but because too simple, resulting in CICD only a single line, but only realized the CI package, not the process of CD with serial up. Is simply, the user clicks on the construction of a mirror just be able to play, but if you want to deploy to kubernetes, or in applications where the need to manually replace it mirrored version. Overall, this version of jenkins we are still using single point, insufficient to support the build than the larger cases, even if the current service hung up, off the net, the whole block building capabilities are not used.


    
    xxx
    
        
            
                
                    buildParam
                    v1
                
                
                    codeBranch
                    master
                
            
        
    
    
        2
        
            
                http://xxxxx.git
                002367566a4eb4bb016a4eb723550054
            
        
        
            
                ${codeBranch}
            
        
        false
        
    
    
        
            ls
        
        
            clean package install -Dmaven.test.skip=true
            mvn3.5.4
        
        
            
                unix:///var/run/docker.sock
            
            
                http://xxxx
            
            xxx/xx
            true
            Dockerfile
            ${buildParam}
            true
        
    
    
        
    

Two, CICD after optimization

The above process still no process did not live DevOps, and human intervention is still a lot of things, due to the urgent need higher output, we can only make do continue. We will build, deploy each as a small piece of a process CICD can choose to build, deploy, spent a lot of energy to complete the serialization of a different kind of CICD. The following diagram, for example, the bottom of the whole process is: paas platform -jenkins-kakfa- management platform (select cicd next) -kafka-cicd component calls management platform trigger (select cicd next step in building -jenkins-kafka- management platform ) -kafka-cicd component invokes the trigger deployment management platform.

Currently realized serialized CICD build deployment, after considering implementing multiple CICD parallel, and a CICD can call another CICD, the actual operation, there has been a lot of problems. Because too many components have been the primary cicd run error, it is difficult to troubleshoot the cause of the problem, the business side of complaints began more slowly, can only say to persuade them not to use this feature.

No CICD, it can not help cloud container, the container can not be a reasonable use of the characteristics of cloud on the company, but can not embark on road cloud-native. So, we decided to find another way out.

Third, the survey period

So many before CICD issue, especially after too many components, resulting in a problem when the investigation can not be normal, to be more stable and reliable, it was decided to replace the bottom. We re-examined under the pipeline, I think this is the right thing to do, but I do not know if things make a product look like, how will the user side Dockerfile do not write, you let him write a Jenkinsfile? unreasonable! In addition to this, we see a serverless jenkins, Google’s tekton. Gitlab comes GitLab-CICD tools cicd, the need to configure the runner, and then configure it .gitlab-ci.yml cicd process to write about the program, when building a mirrored we use is kaniko, in whole gitlab of cicd after our little project, large-scale use, but the high cost of learning, especially the introduction of kaniko, or looking for a product of CICD program.

Jenkins x distributed build must first solve the problem is to build multiple simultaneously running, long before the research had jenkins x, it must be used on kubernetes, because at that time the official documentation was incomplete, and our DevOps project is in the initial phase, all did not use. jenkins the master slave structure will not say more. jenkins x should be said that a family bucket, includes helm warehouse, nexus warehouse, docker registry and so on, the code is jenkins-x-image.

serverless jenkins like with Google’s tekton associated with the next, did not transfer through, only for GitHub. Feeling not as directly tekton.

Ali and so effect provides a graphical configuration DevOps processes, supports timer trigger, but did not trigger combined with gitlab, if you need a firm level of DevOps, requires the company jira, gitlab, jenkins, etc. together, but graphical jenkins pipeline is a special good reference direction, and so efficiency can be combined with Ali to do a DevOps own products.

Microsoft also offers Microsoft Pipeline DevOps solutions, but also provides a written yaml format, namely: the right after filling out will be converted into yaml. If you want to fight the DevOps a product, this design is obviously not the best.

Google tekton official cicd kubernetes, has been used kubernetes a release issued version of the process, at present it is only in conjunction with GitHub, gitlab can not be used, the whole process can be used yaml file to create, run up it is the same as similar kubernetes the job, with That is complete destruction, but at present relatively new, still in alpha version, can not be used for production. Are interested can refer to the following: Knative first experience: CICD speed entry

Fourth, the product of DevOps platforms

DockOne in the research and DevOps each manufacturer’s products, found that really only Ali cloud cloud effect is really more perfect DevOps products, users do not need to know the syntax of the pipeline does not need to acquire the relevant knowledge kubernetes even without yaml write files, for developing, testing is simply the same God exists. After the cloud effect for small companies (start-ups) is free, but there is a certain amount, we will start charging. After a lot of research cloud effective thing, efficiency is found to cloud-based jenkins x transformation, but Ali after all many people, although they would see is about as grammar pipeline, but Ali completely transformed into yaml can use to interact with the background . The following is Ali cloud cloud interface and pipeline efficiency with jenkins grammar to explain:

4.1 Java Code Scanning

PMD is a static code analyzer can expand it not only can the code analyzer, it can not only check the code style, you can also check the issues of design, multi-threading, and performance. Ali cloud is a simple integrated about it, for us, the bottom using sonar to access all of the code have access to the results of the scan sonar.

stage('Clone') {
    steps{
        git branch: 'master', credentialsId: 'xxxx', url: "xxx"
    }
}
stage('check') {
    steps{
        container('maven') {
            echo "mvn pmd:pmd"
        }
    }
}

4.2 Java unit testing

Java unit testing is generally used in Junit, Ali cloud, using a surefire plug-in test phase for the implementation of maven build lifecycle of an application of unit testing. It will have two different forms of the test results. I am here simply too little, using “mvn test” command instead.


stage('Clone') {
    steps{
        echo "1.Clone Stage"
        git branch: 'master', credentialsId: 'xxxxx', url: "xxxxxx"
    }
}
stage('test') {
    steps{
        container('maven') {
            sh "mvn test"
        }
    }
}

4.3 Java build and upload image

Construction of mirrored compare want to use kaniko, try to find a lot of methods to eventually use only dind (docker in docker), Mount docker host to build, if there are other programs, hoping to remind. Currently jenkins x using dind, mount when the need to configure config.json, then mount /root/.docker directory container, in order to use docker in the container.

Why not recommend dind: mount a docker host, you can view a container using a docker ps is running, it means you can use docker stop, docker rm to control the host vessel, although kubernetes will reschedule them, but this section of the restart time a great impact on business.


stage('下载代码') {
    steps{
        echo "1.Clone Stage"
        git branch: 'master', credentialsId: 'xxxxx', url: "xxxxxx"
        script {
            build_tag = sh(returnStdout: true, script: 'git rev-parse --short HEAD').trim()
        }
    }
}
stage('打包并构建镜像') {
    steps{
        container('maven') {
            echo "3.Build Docker Image Stage"
            sh "mvn clean install -Dmaven.test.skip=true"
            sh "docker build -f xxx/Dockerfile -t xxxxxx:${build_tag} ."
            sh "docker push xxxxxx:${build_tag}"
        }
    }
}

4.4 deployed to Ali cloud k8s

CD process is a bit difficult, because of our kubernetes platform is a graphical, similar to Ali cloud, users do not need to write their own deployment, only need to do some operation to be deployed in the graphical interface. For the CD process, if the application exists, then it can be a direct replacement swap can be mirrored version, if there is no application, provides a simple interface that allows users to create a new application. Of course, at the time of the initial implementation of the container, for the user, all of a sudden need to accept docker, kubernetes, helm concepts is very difficult, not one by one to help them write deployment yaml these files can only be created by a common spring helm boot or other template, and then let the business side to modify their own configuration, each time built only need to replace the mirror.

def tmp = sh (
    returnStdout: true,
    script: "kubectl get deployment -n ${namespace} | grep ${JOB_NAME} | awk '{print \$1}'"
)
//如果是第一次,则使用helm模板创建,创建完后需要去epaas修改pod的配置
if(tmp.equals('')){
    sh "helm init --client-only"
    sh """helm repo add mychartmuseum http://xxxxxx \
                       --username myuser \
                       --password=mypass"""
    sh """helm install --set name=${JOB_NAME} \
                       --set namespace=${namespace} \
                       --set deployment.image=${image} \
                       --set deployment.imagePullSecrets=${harborProject} \
                       --name ${namespace}-${JOB_NAME} \
                       mychartmuseum/soa-template"""
}else{
    println "已经存在,替换镜像"
    //epaas中一个pod的容器名称需要带上"-0"来区分
    sh "kubectl set image deployment/${JOB_NAME} ${JOB_NAME}-0=${image} -n ${namespace}"
}

4.5 overall process

Scanning the code, unit testing, run in parallel to build three mirror, three, etc. After completion, during deployment

pipeline:

pipeline {
    agent {
        label "jenkins-maven"
    }
    stages{
        stage('代码扫描,单元测试,镜像构建'){
            parallel {
                stage('并行任务一') {
                    agent {
                        label "jenkins-maven"
                    }
                    stages('Java代码扫描') {
                        stage('Clone') {
                            steps{
                                git branch: 'master', credentialsId: 'xxxxxxx', url: "xxxxxxx"
                            }
                        }
                        stage('check') {
                            steps{
                                container('maven') {
                                    echo "$BUILD_NUMBER"
                                }
                            }
                        }
                    }
                }
                stage('并行任务二') {
                    agent {
                        label "jenkins-maven"
                    }
                    stages('Java单元测试') {
                        stage('Clone') {
                            steps{
                                echo "1.Clone Stage"
                                git branch: 'master', credentialsId: 'xxxxxxx', url: "xxxxxxx"
                            }
                        }
                        stage('test') {
                            steps{
                                container('maven') {
                                    echo "3.Build Docker Image Stage"
                                    sh "mvn -v"
                                }
                            }
                        }
                    }
                }
                stage('并行任务三') {
                    agent {
                        label "jenkins-maven"
                    }
                    stages('java构建镜像') {
                        stage('Clone') {
                            steps{
                                echo "1.Clone Stage"
                                git branch: 'master', credentialsId: 'xxxxxxx', url: "xxxxxxx"
                                script {
                                    build_tag = sh(returnStdout: true, script: 'git rev-parse --short HEAD').trim()
                                }
                            }
                        }
                        stage('Build') {
                            steps{
                                container('maven') {
                                    echo "3.Build Docker Image Stage"
                                    sh "mvn clean install -Dmaven.test.skip=true"
                                    sh "docker build -f epaas-portal/Dockerfile -t hub.gcloud.lab/rongqiyun/epaas:${build_tag} ."
                                    sh "docker push hub.gcloud.lab/rongqiyun/epaas:${build_tag}"
                                }
                            }
                        }
                    }
                }
            }
        }
        stage('部署'){
            stages('部署到容器云') {
                stage('check') {
                    steps{
                        container('maven') {
                            script{
                                if (deploy_app == "true"){
                                    def tmp = sh (
                                        returnStdout: true,
                                        script: "kubectl get deployment -n ${namespace} | grep ${JOB_NAME} | awk '{print \$1}'"
                                    )
                                    //如果是第一次,则使用helm模板创建,创建完后需要去epaas修改pod的配置
                                    if(tmp.equals('')){
                                        sh "helm init --client-only"
                                        sh """helm repo add mychartmuseum http://xxxxxx \
                                                           --username myuser \
                                                           --password=mypass"""
                                        sh """helm install --set name=${JOB_NAME} \
                                                           --set namespace=${namespace} \
                                                           --set deployment.image=${image} \
                                                           --set deployment.imagePullSecrets=${harborProject} \
                                                           --name ${namespace}-${JOB_NAME} \
                                                           mychartmuseum/soa-template"""
                                    }else{
                                        println "已经存在,替换镜像"
                                        //epaas中一个pod的容器名称需要带上"-0"来区分
                                        sh "kubectl set image deployment/${JOB_NAME} ${JOB_NAME}-0=${image} -n ${namespace}"
                                    }
                                }else{
                                    println "用户选择不部署代码"
                                }
                            }
                        }
                    }
                }
            }
        }
    }
}

Check in at jenkins x:

4.4 log

jenkins blue ocean log steps:

Cloud log in effect:

4.5 Timing trigger

    triggers {
        cron('H H * * *') //每天
    }

V. Other

5.1 Gitlab trigger

pipeline in addition to the trigger time, also supports the gitlab trigger, you need a variety of configurations, but if you really have to gitlab of cicd requirements directly gitlab-ci will be better, we also carried out on the runner’s gitlab configured to support gitlab of cicd. gitlab of cicd also provides a process that is constructed after the destruction.

VI Summary

The most powerful process than their use scripts to achieve pipeline, select the most suitable for them, but for a company, if required to master the business side, especially when IT mobility, both need to re-training, with questions will be asked many times, so, DevOps can only be implemented as a graphical thing, convenient, simple, relatively speaking, fairly powerful features.

DevOps may not be the most difficult of these, the key is to get users to accept, when the container is first cloud implementation, the company had a lot of hair version of the traditional ways need to be changed, and some do not want to change the business side, some of the code or the persistence of things to keep the code rather than distributed storage, and even some users are reluctant party to maintain old code can not even look and then think on the container, a company doing technical architecture of the time, too confusing to finally fill the hole either takes too much energy even big shake.

Finally, DevOps cloud is the only way native! ! !

文章同步:
博客园:https://www.cnblogs.com/w1570631036/p/11524673.html
个人网站:http://www.wenzhihuai.com/getblogdetail.html?blogid=663
gitbook:https://gitbook.wenzhihuai.com/devops/devops-ping-tai

Leave a Reply