🗒️Jenkins pipeline实验
2023-4-22
| 2023-4-23
0  |  0 分钟
type
status
date
slug
summary
tags
category
icon
password
本实验基于以下前提：
  • 已有kubernetes集群，创建了相关secret
  • Jenkins kubernetes相关插件已配置好，pod template要留空
  • 网络条件OK
  • 已具备storage class，如果没有，需要改挂载
  • 已配置settings.xml，可以加速Maven构建
实验仅仅是为了验证多container的可行性，故步骤很简单。没有kubernetes插件之前，job实现流水线基本都依赖Jenkins master的插件功能，比如在系统菜单配置凭据供流水线调用，在系统菜单配置maven等插件供流水线build。有了kubernetes插件，凭据可以使用secret部署在Jenkins slave pod运行的namespace中（可以对此namespace做权限控制，secret泄露），maven等插件变成了slave pod中的一个container，可以实现特定版本号、定制化需求等。
范例的Git仓库是公开的，不需要密钥，更多细节见下面。

1. 实验pipeline

用到的Settings.xml，配置了阿里云仓库
<?xml version="1.0" encoding="UTF-8"?>

<settings xmlns="http://maven.apache.org/SETTINGS/1.0.0" 
          xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" 
          xsi:schemaLocation="http://maven.apache.org/SETTINGS/1.0.0 http://maven.apache.org/xsd/settings-1.0.0.xsd">

  <pluginGroups>
  </pluginGroups>

  <proxies>
  </proxies>

  <servers>
  </servers>
  
  <mirrors>
    <!--Aliyun Maven-->
    <mirror>
        <id>alimaven</id>
        <name>aliyun maven</name>
        <url>http://maven.aliyun.com/nexus/content/groups/public/</url>
        <mirrorOf>central</mirrorOf>
    </mirror>
  </mirrors>
  
  <profiles>
  </profiles>

</settings>
def label = "jenkins-slave-${UUID.randomUUID().toString()}" 
//设置超时
timeout(time: 600, unit: 'SECONDS') {
    podTemplate(label: label, cloud: 'kubernetes', 
  containers: [
    containerTemplate(name: 'maven', image: 'docker.io/library/maven:latest', ttyEnabled: true, command: 'cat'),
     //k8s-kubectl未写tag，故拉取的是最新版，log里的version可以看到版本
    containerTemplate(name: 'kubectl', image: 'lachlanevenson/k8s-kubectl', command: 'cat', ttyEnabled: true)],
  volumes: [
    persistentVolumeClaim(mountPath: '/root/.m2', claimName: 'jenkins-m2'),
    persistentVolumeClaim(mountPath: '/home/jenkins/agent/workspace', claimName: 'jenkins-agent'),
    //jenkins-k8s-cfg和jenkins-k8s-realip的相同点：都是一个集群的$home/.kube/config文件
    //jenkins-k8s-cfg和jenkins-k8s-realip的不同点：sever连接串不一样，jenkins-k8s-cfg是集群内连接串https://kubernetes.default.svc.cluster.local，jenkins-k8s-realip是https://节点IP:6443。
    //节点IP的连接串适用范围更广，因为可以联通别的集群。
    //secretVolume(secretName: 'jenkins-k8s-cfg', mountPath: '/home/jenkins/agent/.kube'),
    secretVolume(secretName: 'jenkins-k8s-realip', mountPath: '/home/jenkins/agent/.kube')]
  ){
  node(label){
    stage('Build'){
        //本地连公网Git非常慢，所以用了代理。真实环境都是内部gitlab地址，一般不需要代理。
        sh 'git config --global http.https://github.com.proxy http://192.168.1.2:7890'
        sh 'git config -l'
        git branch: 'master', url: 'https://github.com/my-dlq/springboot-helloworld.git'
        container('maven') {
            //已提前配置好settings.xml
            configFileProvider([configFile(fileId: "global-maven-settings", targetLocation: "settings.xml")]){
                stage('Build a Maven project') {
                    sh 'mvn clean package -Dfile.encoding=UTF-8 -DskipTests=true'
                }
            }
        }
    }
    stage('kubectl'){
      container('kubectl') {
        //sh 'pwd;ls -l;ps aux'
        sh 'kubectl version' 
        sh 'ls -l /home/jenkins/agent/.kube/'
        //configRealip4jenki是创建secret时--from-file文件名
        sh 'kubectl --kubeconfig=/home/jenkins/agent/.kube/configRealip4jenkins get pod -n demo'
      }
    }
   }
  } 
}

2. 实验log

Started by user admin
[Pipeline] Start of Pipeline
[Pipeline] timeout
Timeout set to expire in 10 min
[Pipeline] {
[Pipeline] podTemplate
[Pipeline] {
[Pipeline] node
Created Pod: kubernetes demo/jenkins-slave-42edac3f-677b-4fd2-aa7a-3d38f95cf244-9nx3l-kgbkm
Agent jenkins-slave-42edac3f-677b-4fd2-aa7a-3d38f95cf244-9nx3l-kgbkm is provisioned from template jenkins-slave-42edac3f-677b-4fd2-aa7a-3d38f95cf244-9nx3l
---
apiVersion: "v1"
kind: "Pod"
metadata:
  annotations:
    buildUrl: "http://10.110.43.229:8080/job/dul-container/26/"
    runUrl: "job/dul-container/26/"
  labels:
    jenkins: "slave"
    jenkins/label-digest: "2e6beab17f92d73e7c1a00642056379c66e06e5d"
    jenkins/label: "jenkins-slave-42edac3f-677b-4fd2-aa7a-3d38f95cf244"
  name: "jenkins-slave-42edac3f-677b-4fd2-aa7a-3d38f95cf244-9nx3l-kgbkm"
  namespace: "demo"
spec:
  containers:
  - command:
    - "cat"
    image: "docker.io/library/maven:latest"
    imagePullPolicy: "IfNotPresent"
    name: "maven"
    resources: {}
    tty: true
    volumeMounts:
    - mountPath: "/root/.m2"
      name: "volume-0"
      readOnly: false
    - mountPath: "/home/jenkins/agent/workspace"
      name: "volume-1"
      readOnly: false
    - mountPath: "/home/jenkins/agent/.kube"
      name: "volume-2"
      readOnly: false
    #$workdir挂载了emptyDIR
    - mountPath: "/home/jenkins/agent"
      name: "workspace-volume"
      readOnly: false
  - command:
    - "cat"
    image: "lachlanevenson/k8s-kubectl"
    imagePullPolicy: "IfNotPresent"
    name: "kubectl"
    resources: {}
    tty: true
    volumeMounts:
    - mountPath: "/root/.m2"
      name: "volume-0"
      readOnly: false
    - mountPath: "/home/jenkins/agent/workspace"
      name: "volume-1"
      readOnly: false
    - mountPath: "/home/jenkins/agent/.kube"
      name: "volume-2"
      readOnly: false
    #$workdir挂载了emptyDIR
    - mountPath: "/home/jenkins/agent"
      name: "workspace-volume"
      readOnly: false
  - env:
    - name: "JENKINS_SECRET"
      value: "********"
    - name: "JENKINS_TUNNEL"
      value: "10.110.43.229:50000"
    - name: "JENKINS_AGENT_NAME"
      value: "jenkins-slave-42edac3f-677b-4fd2-aa7a-3d38f95cf244-9nx3l-kgbkm"
    - name: "JENKINS_NAME"
      value: "jenkins-slave-42edac3f-677b-4fd2-aa7a-3d38f95cf244-9nx3l-kgbkm"
    - name: "JENKINS_AGENT_WORKDIR"
      value: "/home/jenkins/agent"
    - name: "JENKINS_URL"
      value: "http://10.110.43.229:8080/"
    #inbound-agent容器是插件默认启动的，我并没有配置
    image: "jenkins/inbound-agent:3107.v665000b_51092-5"
    name: "jnlp"
    resources:
      requests:
        memory: "256Mi"
        cpu: "100m"
    volumeMounts:
    - mountPath: "/root/.m2"
      name: "volume-0"
      readOnly: false
    - mountPath: "/home/jenkins/agent/workspace"
      name: "volume-1"
      readOnly: false
    - mountPath: "/home/jenkins/agent/.kube"
      name: "volume-2"
      readOnly: false
    #$workdir挂载了emptyDIR
    - mountPath: "/home/jenkins/agent"
      name: "workspace-volume"
      readOnly: false
  nodeSelector:
    kubernetes.io/os: "linux"
  restartPolicy: "Never"
  volumes:
  - name: "volume-0"
    persistentVolumeClaim:
      claimName: "jenkins-m2"
      readOnly: false
  - name: "volume-2"
    secret:
      secretName: "jenkins-k8s-cfg"
  - name: "volume-1"
    persistentVolumeClaim:
      claimName: "jenkins-agent"
      readOnly: false
  - emptyDir:
      medium: ""
    name: "workspace-volume"

Running on jenkins-slave-42edac3f-677b-4fd2-aa7a-3d38f95cf244-9nx3l-kgbkm in /home/jenkins/agent/workspace/dul-container
[Pipeline] {
[Pipeline] stage
[Pipeline] { (Build)
[Pipeline] sh
+ git config --global http.https://github.com.proxy http://192.168.1.2:7890
[Pipeline] sh
+ git config -l
filter.lfs.clean=git-lfs clean -- %f
filter.lfs.smudge=git-lfs smudge -- %f
filter.lfs.process=git-lfs filter-process
filter.lfs.required=true
http.https://github.com.proxy=http://192.168.123.2:7890
core.repositoryformatversion=0
core.filemode=true
core.bare=false
core.logallrefupdates=true
core.hookspath=/dev/null
remote.origin.url=https://github.com/my-dlq/springboot-helloworld.git
[Pipeline] git
The recommended git tool is: NONE
No credentials specified
Fetching changes from the remote Git repository
 > git rev-parse --resolve-git-dir /home/jenkins/agent/workspace/dul-container/.git # timeout=10
 > git config remote.origin.url https://github.com/my-dlq/springboot-helloworld.git # timeout=10
Checking out Revision 86bba333f546ac67b83a2dea9b029f08358d9f75 (refs/remotes/origin/master)
Fetching upstream changes from https://github.com/my-dlq/springboot-helloworld.git
 > git --version # timeout=10
 > git --version # 'git version 2.30.2'
 > git fetch --tags --force --progress -- https://github.com/my-dlq/springboot-helloworld.git +refs/heads/*:refs/remotes/origin/* # timeout=10
 > git rev-parse refs/remotes/origin/master^{commit} # timeout=10
 > git config core.sparsecheckout # timeout=10
 > git checkout -f 86bba333f546ac67b83a2dea9b029f08358d9f75 # timeout=10
 > git branch -a -v --no-abbrev # timeout=10
 > git branch -D master # timeout=10
 > git checkout -b master 86bba333f546ac67b83a2dea9b029f08358d9f75 # timeout=10
Commit message: "修改配置"
 > git rev-list --no-walk 86bba333f546ac67b83a2dea9b029f08358d9f75 # timeout=10
[Pipeline] container
[Pipeline] {
[Pipeline] configFileProvider
provisioning config files...
copy managed file [MyGlobalMavenSettings] to file:/home/jenkins/agent/workspace/dul-container/settings.xml
[Pipeline] {
[Pipeline] stage
[Pipeline] { (Build a Maven project)
[Pipeline] sh
+ mvn clean package -Dfile.encoding=UTF-8 -DskipTests=true
[INFO] Scanning for projects...
[INFO] 
[INFO] ------------------< club.mydlq:springboot-helloworld >------------------
[INFO] Building springboot-helloworld 0.0.1
[INFO]   from pom.xml
[INFO] --------------------------------[ jar ]---------------------------------
[INFO] 
[INFO] --- clean:3.1.0:clean (default-clean) @ springboot-helloworld ---
[INFO] Deleting /home/jenkins/agent/workspace/dul-container/target
[INFO] 
[INFO] --- resources:3.1.0:resources (default-resources) @ springboot-helloworld ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] Copying 1 resource
[INFO] Copying 0 resource
[INFO] 
[INFO] --- compiler:3.8.0:compile (default-compile) @ springboot-helloworld ---
[INFO] Changes detected - recompiling the module!
[INFO] Compiling 2 source files to /home/jenkins/agent/workspace/dul-container/target/classes
[INFO] 
[INFO] --- resources:3.1.0:testResources (default-testResources) @ springboot-helloworld ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] skip non existing resourceDirectory /home/jenkins/agent/workspace/dul-container/src/test/resources
[INFO] 
[INFO] --- compiler:3.8.0:testCompile (default-testCompile) @ springboot-helloworld ---
[INFO] No sources to compile
[INFO] 
[INFO] --- surefire:2.22.1:test (default-test) @ springboot-helloworld ---
[WARNING] Parameter 'localRepository' is deprecated core expression; Avoid use of ArtifactRepository type. If you need access to local repository, switch to '${repositorySystemSession}' expression and get LRM from it instead.
[INFO] Tests are skipped.
[INFO] 
[INFO] --- jar:3.1.1:jar (default-jar) @ springboot-helloworld ---
[INFO] Building jar: /home/jenkins/agent/workspace/dul-container/target/springboot-helloworld-0.0.1.jar
[INFO] 
[INFO] --- spring-boot:2.1.4.RELEASE:repackage (repackage) @ springboot-helloworld ---
[INFO] Replacing main artifact with repackaged archive
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  3.946 s
[INFO] Finished at: 2023-04-22T12:18:27Z
[INFO] ------------------------------------------------------------------------
[Pipeline] }
[Pipeline] // stage
[Pipeline] }
[Pipeline] // configFileProvider
[Pipeline] }
[Pipeline] // container
[Pipeline] }
[Pipeline] // stage
[Pipeline] stage
[Pipeline] { (kubectl)
[Pipeline] container
[Pipeline] {
[Pipeline] sh
+ kubectl version
WARNING: This version information is deprecated and will be replaced with the output from kubectl version --short.  Use --output=yaml|json to get the full version.
Client Version: version.Info{Major:"1", Minor:"25", GitVersion:"v1.25.4", GitCommit:"872a965c6c6526caa949f0c6ac028ef7aff3fb78", GitTreeState:"clean", BuildDate:"2022-11-09T13:36:36Z", GoVersion:"go1.19.3", Compiler:"gc", Platform:"linux/amd64"}
Kustomize Version: v4.5.7
Server Version: version.Info{Major:"1", Minor:"24", GitVersion:"v1.24.10", GitCommit:"5c1d2d4295f9b4eb12bfbf6429fdf989f2ca8a02", GitTreeState:"clean", BuildDate:"2023-01-18T19:08:10Z", GoVersion:"go1.19.5", Compiler:"gc", Platform:"linux/amd64"}
[Pipeline] sh
+ ls -l /home/jenkins/agent/.kube/config4jenkins
lrwxrwxrwx    1 root     root            21 Apr 22 12:18 /home/jenkins/agent/.kube/config4jenkins -> ..data/config4jenkins
[Pipeline] sh
+ kubectl '--kubeconfig=/home/jenkins/agent/.kube/config4jenkins' get pod -n demo
NAME                                                             READY   STATUS    RESTARTS      AGE
jenkins-776d6f7d88-pbxr4                                         1/1     Running   1 (75m ago)   31h
jenkins-slave-42edac3f-677b-4fd2-aa7a-3d38f95cf244-9nx3l-kgbkm   3/3     Running   0             19s
[Pipeline] }
[Pipeline] // container
[Pipeline] }
[Pipeline] // stage
[Pipeline] }
[Pipeline] // node
[Pipeline] }
[Pipeline] // podTemplate
[Pipeline] }
[Pipeline] // timeout
[Pipeline] End of Pipeline
Finished: SUCCESS
相关文章 :
技术
  • k8s
  • devops
    • Jenkins scripted pipeline使用sshagent和参数Jenkins pod插件相关
    目录