Multibranch Pipeline

Problems

Like database product, it runs on multi-platform, but for software enginner they may only works on one platform, how they could identify their code works on all platform? manually build the various platforms? NO!

Solution

Most people would know we can use Jenkins pipeline, they may create multi Jenkins job for different stuation.

How to do it in an elegant way, I would want to share how to use multibranch pipeline to achieve.

  1. When create a pull request, auto parallel start simple build.
  2. Reviewers can decide whether to merge base on build results.
  3. After code merged, auto start full build.

Read More

A Code Coverage Tool - Squish Coco use examples

Preparation

  1. You need to ask for a free trial license and install
    • You will receive a mail with username/password to login for downloading
  2. I test it on the Windows platform, so I download the Windows installer, then install Squish Coco and Add-in
  3. Installed Visual Studio 2010 or higher, I used VS2017 Professional

Read More

Code Coverage tools of C/C++

Code Coverage is a measurement of how many lines, statements, or blocks of your code are tested using your suite of automated tests. It’s an essential metric to understand the quality of your QA efforts.

Code coverage shows you how much of your application is not covered by automated tests and is therefore vulnerable to defects. it is typically measured in percentage values – the closer to 100%, the better.

When you’re trying to demonstrate test coverage to your higher-ups, code coverage tools (and other tools of the trade, of course) come in quite useful.

List of Code Coverage Tools

| Tools | Support Language | Cost | Partners |
|—|—|—|—|—|
| Squish Coco | C, C++, C#, SystemC, Tcl and QML | Not disclosed |Botom of this page Selected Clients|
| BullseyeCoverage | C, C++ |$800 for 1-year license and up | |
| Testwell| C, C++, C#, Java| Not disclosed| |
| Parasoft C/C++test |C, C++ | Not disclosed | partners |
| VECTOR Code Coverage | C, C++ | Not disclosed (free trial available)| partners |
|JaCoCo| Java | Open Source| Most famous code coverage tool in Java area |

GitSCM clone code don't display branch

最近遇到一个 regression bug,是产品完成构建之后,build commit number 不对,显示的 HEAD 而不是常见的 97b34931ac HASH number,这是什么原因呢?
我检查了 build 脚本没有发现问题,branch 的输出是正确的,那我怀疑是引入 Jenkins 的原因,果然登录到远程的 agent 上去查看分支名称如下:

C:\workspace\blog>git branch
* (HEAD detached at 97b3493)

果然问题出在了 Jenkins 上。这个问题有简单办法解决,就是直接使用git命令来clone代码,而不使用Git插件

git clone --depth 1 -b u2opensrc https://username:"passwowrd"@git.github.com/scm/blog.git blog

这种方式固然简单,不会出错,但它是明码显示,我岂能容忍这种不堪的处理方式吗?肯定还是要在 Git 插件上找到解决办法的。
随后google一下,果然有遇到和我一样问题的人,问题链接 这里

他说他做了很多调查,还跟专业的 Jenkins 人士联系,试了很多次,最后找到这个办法

checkout([$class: 'GitSCM', branches: [[name: '*/feature/*']], doGenerateSubmoduleConfigurations: false, extensions: [[$class: 'LocalBranch', localBranch: "**"]], submoduleCfg: [], userRemoteConfigs: [[credentialsId: '99f978af-XXXX-XXXX-8147-2cf8f69ef864', url: 'http://TFS_SERVER:8080/tfs/DefaultCollection/Product/_git/Project']]])

主要是在 extensions:[] 中加入这句 [$class: ‘LocalBranch’, localBranch: “**”]

这是 Jenkins 的 Bug 吗?带着这个疑问随后通过 Pipeline Syntax,找到 checkout: Check out from version control,在 Additional Behaviours 里有 Check out to specific local branch 这个配置项

If given, checkout the revision to build as HEAD on this branch.
If selected, and its value is an empty string or “**”, then the branch name is computed from the remote branch without the origin. In that case, a remote branch origin/master will be checked out to a local branch named master, and a remote branch origin/develop/new-feature will be checked out to a local branch named develop/newfeature.

看介绍原来 Jenkins 自带这个设置,只是它不是默认选项,所以才遇到刚才那个问题。随后选择这个设置,然后填入”**”,然后生成 Pipeline 脚本,就跟上面的脚本一样了。

通过参数化上传文件到 FTP 服务器

实现 CI/CD 过程中,常常需要将构建好的 build 上传到一个公共的服务器,供测试、开发来获取最新的 build。如何上传 build 成果物到 FTP server,又不想把 FTP server登录的用户名和密码存在脚本里,想做这样的参数化如何实现呢?

upload_to_ftp.bat [hostname] [username] [password] [local_path] [remote_pat]

Read More

Automatically commit code by Jenkins

When we need to release a product, we should change copyright, build version, release month, release note…
How to modify multiple files automatically?
I used a Jenkins pipeline project, the project is parameterized(string parameter) and regular expressions to implement.

  1. Here is the string parameter for copyright:

    • Name: copyright

    • Default Value: 1995—2019

    • Description: Copyright format:1995—2019

      stage('change copyrigh') {
      steps {
      sh label: '', script: 'sed -i -E "s/(1995—[0-9]{4})/${copyright}/" 1033/AutoRun.ini'
      }
      }
  2. Here is the string parameter for build version:

    • Name: build_version

    • Default Value: 1.2.2.1002

    • Description: build version format: 1.2.2.1002

      stage('change build version') {
      steps {
      sh label: '', script: 'sed -i -E "s/([0-9].[0-9].[0-9].[0-9]{4})/${build_version}/" 1033/AutoRun.ini'
      }
      }
  3. Here is the string parameter for build version:

    • Name: release_month

    • Default Value: May 2019

    • Description: release month format: May 2019

      stage('change release month') {
      steps {
      sh label: '', script: '''
      sed -i -E "s/([a-z]* 20[0-9]{2})/${release_month}/" 1033/AutoRun.ini
      sed -i -E "s/([a-z]* 20[0-9]{2})/${release_month}/" 1033/MainMenu.ini
      '''
      }
      }
  4. push change to Git

    stage('git push to Git') {
    steps {
    sshagent(['8dd766ba-ac0f-4302-afa8-bee59c726dee']) {
    sh("git add 1033/AutoRun.ini")
    sh("git add 1033/MainMenu.ini")
    sh("git commit -m 'Bld # ${build_version}'")
    sh("git push -u origin master")
    }
    }
    }
  5. Whole Jenkins Pipeline looks like:

    pipeline {
    agent {
    label 'master'
    }

    stages{
    stage('git clone') {
    steps{
    git branch: 'master',
    credentialsId: '8dd766ba-ac0f-4302-afa8-bee59c726dee',
    url: 'git@github.com:shenxianpeng/blog.git'
    }
    }

    stage('change copyrigh') {
    steps {
    sh label: '', script: 'sed -i -E "s/(1995—[0-9]{4})/${copyright}/" 1033/AutoRun.ini'
    }
    }

    stage('change release month') {
    steps {
    sh label: '', script: '''
    sed -i -E "s/([a-z]* 20[0-9]{2})/${release_month}/" 1033/AutoRun.ini
    sed -i -E "s/([a-z]* 20[0-9]{2})/${release_month}/" 1033/MainMenu.ini
    '''
    }
    }

    stage('change build version') {
    steps {
    sh label: '', script: 'sed -i -E "s/([0-9].[0-9].[0-9].[0-9]{4})/${build_version}/" 1033/AutoRun.ini'
    }
    }

    stage('git push to Git') {
    steps {
    sshagent(['8dd766ba-ac0f-4302-afa8-bee59c726dee']) {
    sh("git add 1033/AutoRun.ini")
    sh("git add 1033/MainMenu.ini")
    sh("git commit -m 'Bld # ${build_version}'")
    sh("git push -u origin master")
    }
    }
    }
    }
    }

Jenkins Linux agent configuration

Prepare Java runtime

Check if had installed java

$ java -version
openjdk version "1.8.0_65"
OpenJDK Runtime Environment (build 1.8.0_65-b17)
OpenJDK 64-Bit Server VM (build 25.65-b01, mixed mode)

if not Here is an article telling you how to install it

Create Node

1. Jenkins home page->Manage Node->New Node, such as window-build-machine

2. List Linux agent settings

Items Settings
Name Linux-build-machine
Description used for Linux build
of executors 1
Remote root directory /home/agent
Labels Linux, build
Usage Use this node as much as possible
Launch method Launch agent agents via SSH
Host 192.168.1.112
Credentials username/password
Host Key Verification Strategy Manually trusted key Verification Strategy
Availability Keep this agent online as much as paossible

3. How to set credentials

credentials configuration
Domain Global credentials (unrestricted)
Kind Username with password
Scope Global(Jenkins, nodes, items, all child items, etc)
Username root
Password mypassword
Description Linux agent username & password

4. Save then Connect

Remoting version: 3.29
This is a Unix agent
Evacuated stdout
Agent successfully connected and online
SSHLauncher{host='192.168.1.112', port=22, credentialsId='d1cbab74-823d-41aa-abb7-8584859503d0', jvmOptions='', javaPath='/usr/bin/java',
prefixStartSlaveCmd='', suffixStartSlaveCmd='', launchTimeoutSeconds=210, maxNumRetries=10, retryWaitTime=15,
sshHostKeyVerificationStrategy=hudson.plugins.sshslaves.verifiers.ManuallyTrustedKeyVerificationStrategy, tcpNoDelay=true, trackCredentials=true}
[05/11/19 01:33:37] [SSH] Opening SSH connection to 192.168.1.112:22.
[05/11/19 01:33:37] [SSH] SSH host key matches key seen previously for this host. Connection will be allowed.
[05/11/19 01:33:37] [SSH] Authentication successful.
[05/11/19 01:33:37] [SSH] The remote user's environment is:

Troubleshooting

Problem how to fix
[04/22/19 23:15:07] [SSH] WARNING: No entry currently exists in the Known Hosts file for this host. Connections will be denied until this new host and its associated key is added to the Known Hosts file. ssh-keyscan HOSTNAME >> known_hosts
/var/lib/jenkins/.ssh/known_hosts [SSH] No Known Hosts file was found at /var/lib/jenkins/.ssh/known_hosts. changing the Host key verification strategy in LAUNCH METHOD from “Known Hosts file verification strategy” to “Manually trusted key verification strategy”

Jenkins Windows agent configuration

Prepare Java runtime

1. Download Java

2. Configure Java Windows path

JAVA_HOME=C:\Program Files\Java\jdk1.8.0_201
CLASSPATH=.;%JAVA_HOME%\lib;%JAVA_HOME%\jre\lib

Create Node

1. Jenkins home page->Manage Node->New Node, such as window-build-machine

2. List windows agent settings

Items Settings
Name window-build-machine
Description used for windows build
of executors 1
Remote root directory C:\agent
Labels windows, build
Usage Use this node as much as possible
Launch method Let Jenkins control this Windows slave as a Windows service
Administrator user name .\Administrator
Password mypassword
Host 192.168.1.111
Run service as Use Administrator account given above
Availability Keep this agent online as much as paossible

3. Save then Connect

[2019-05-11 01:32:50] [windows-slaves] Connecting to 192.168.1.111
Checking if Java exists
java -version returned 1.8.0.
[2019-05-11 01:32:50] [windows-slaves] Copying jenkins-slave.xml
[2019-05-11 01:32:50] [windows-slaves] Copying slave.jar
[2019-05-11 01:32:50] [windows-slaves] Starting the service
[2019-05-11 01:32:50] [windows-slaves] Waiting for the service to become ready
[2019-05-11 01:32:55] [windows-slaves] Connecting to port 52,347
<===[JENKINS REMOTING CAPACITY]===>Remoting version: 3.29
This is a Windows agent
Agent successfully connected and online

Troubleshooting

The following issues I met and how I fixed them.

1. ERROR: Message not found for errorCode: 0xC00000AC

You need need to install JDK, and config JAVA environment variable.

2. How to fix add windows node as Windows service error

Ref to JENKINS-16418.

3. org.jinterop.dcom.common.JIException: Message not found for errorCode: 0x00000005

Fixed permission for the following registry keys

  1. HKEY_LOCAL_MACHINE\SOFTWARE\Classes\Wow6432Node\CLSID{72C24DD5-D70A-438B-8A42-98424B88AFB8}
  2. HKEY_CLASSES_ROOT\CLSID{76A64158-CB41-11D1-8B02-00600806D9B6}

Steps to fix it

  • Open ‘regedit’ (as Administrator), Find (Ctrl+F) the registry key: “{72C24DD5-D70A-438B-8A42-98424B88AFB8}” in HKEY_LOCAL_MACHINE\SOFTWARE\Classes\Wow6432Node\CLSID
  • Right click and select ‘Permissions’, Change owner to administrators group (Advanced…).
  • Change permissions for administrators group. Grant Full Control。
  • Change owner back to TrustedInstaller (user is “NT Service\TrustedInstaller” on local machine)

Repeat the above steps to fix permission for HKEY_CLASSES_ROOT\CLSID{76A64158-CB41-11D1-8B02-00600806D9B6}

Finally, Restart Remote Registry Service (Administrative Tools / Services).

4. ERROR: Unexpected error in launching an agent

This is probably a bug in Jenkins.

  1. Login remote machine and open Services find jenkinsslave-C__agent
  2. Set startup type: Automatic
  3. Log On: select This account, type correct account and password
  4. start jenkinsslave-C__agent

5. Caused by: org.jinterop.dcom.common.JIRuntimeException: Message not found for errorCode: 0x800703FA

Slave under domain account, If your slave is running under a domain account and you get an error code 0x800703FA, change a group policy:

  1. open the group policy editor (gpedit.msc)
  2. go to Computer Configuration->Administrative Templates->System-> UserProfiles, “Do not forcefully unload the user registry at user logoff”
  3. Change the setting from “Not Configured” to “Enabled”, which disables the new User Profile Service feature (‘DisableForceUnload’ is the value added to the registry)

6. ERROR: Message not found for errorCode: 0xC0000001 Caused by: jcifs.smb.SmbException: Failed to connect: 0.0.0.0<00>/10.xxx.xxx.xxx

Need to enable SMB1

  1. Search in the start menu for ‘Turn Windows features on or off’ and open it.
  2. Find ‘SMB1.0/CIFS File Sharing Support’ in the list of optional features that appears, and select the checkbox next to it.
  3. Click OK and Windows will add the selected feature.

You’ll be asked to restart your computer as part of this process.

7. .NET Framework 2.0 or later is required on this computer to run a Jenkins agent as a Windows service

Need to upgrade your .NET Framework. Here is a link for update .NET Framework.

6. more connect jenkins agent problem on windows

Please refer to this link https://github.com/jenkinsci/windows-slaves-plugin/blob/master/docs/troubleshooting.adoc

从测试转开发

每当工作闲暇,我都会时常想起好久没有更新微信公众号的文章了,总想等工作不忙的时候赶紧跟大家分享我从测试转开发这段时间的经历和感受,但工作总是有忙不完的忙,一刻都停不下来。

终于等到这一周有两天工作不是那么忙碌了,才决定将前几天写到一半的文章更新完。这是我这几个月下来感受最轻松的两天,暂时没有bug需要去调查和测试,不用去看十几年的C代码,终于有大块时间去写我负责的Python Client端代码了。这种写着代码,听着歌曲去重构,Debug,修改Unit Test Suite感觉真是幸福。

幸福的时光总是短暂的,今天就又来了两个Bug需要去调查 ε=(´ο`*)))唉…

又把我打回原形,调查大半天之后发现原来是QA测的不对,可以松口气晚上可以不用工作更新下微信公众号了。

这五个月来,几乎每天都是白天八小时,晚上继续背着电脑回家准备继续工作,周日偶尔去公司,经常在家学习。因为角色的转变,新的项目,需要学习的地方很多。从业务到技术,再加上产品发布在即,作为一名开发新人也肩负起Bug Fix的任务,十年前的代码,全英文的文档,复杂的系统,如果不全力一搏,真担心自己转型失败,那就太打脸了。

一天的工作忙碌和压力,使得我晚上总是吃的停不下来,吃饭是我一天当中最轻松的时刻。去年我跟别人打赌减肥赢奖金,我毫无怨念的拿到了第一的奖金,可是今年再和别人打赌减肥,至今我都还没开始,马上年底了,输掉奖金是毫无悬念的。总结下来,大概是因为今年工作太忙,工作压力大的缘故,使得我无法在八小时之余安心去继续练习吉他,做keep,年假还没来得及休,真是计划不如变化快。

虽然我还是个小开发,当角色变了,角度也会有变化。

自动化测试是本分,DevOps是阶梯

这几年下来相信你也会真切感受到,如果一名测试人员不懂自动化测试,不会写自动化测试脚本,不但难有升职或是跳槽的机会,很有可能会被企业所淘汰。

个人觉得DevOps是未来一段时间很多企业要走的路,一般的二线城市能把DevOps讲明白并且实施的人太少了,所以尽早掌握实施DevOps的人,就有机会成为DevOps教练或是测试架构师这样的角色。

没有做好抗压的准备,不要去做开发

这几个月来遇到压力非常多,从刚开始的学习C语言,到C语言考核;从学习全英文的业务文档,到业务文档的分享(也是一种考核);从调研C代码的代码覆盖率、Git分享,到调查并解决Bug;从每天的站立会汇报到每周与国外同事的例会。终于等到九月份,Title从Quality Assurance Engineer变成了Software Engineer,这其中的压力、痛苦和短暂的喜悦只有走过的人才知道。

与年龄想匹配的能力

这点非常重要,如果现在问你,你与刚毕业两三年的同行年轻人有哪些优势?如果你不能肯定和清楚的说出自己优势的话,那就要好好反思一下了。

如果从开发角度来说,我现在就是与年龄不相匹配的能力,因此测试相关的技能以及DevOps相关知识依旧是我要好好掌握的功课。

学好英语

对于国内公司来说,工作上不会用到英语,但我想说如果想在测试和开发领域有更长远发展,英文非常重要。一般最流行开源的自动化测试框架、技术、DevOps相关的工具以及搜索最有效的解决问题的方案一般都是英文。如果你的英语不好,坚持一年半载去硬啃一手英文资料,形成习惯,受益终生。

最受欢迎的 Python 自动化测试框架推荐

随着技术的进步和自动化技术的出现,市面上出现了一些自动化测试框架。只需要进行一些适用性和效率参数的调整,这些自动化测试框架就能够开箱即用,大大节省了开发时间。

本文整理了当前最受欢迎的 Python 自动化测试框架。

Robot Framework

这是最流行的开源 Python 自动化测试框架,表格式的测试数据语法和关键词驱动测试使得它在全球的测试人员中非常流行。它还拥有众多可用的工具和库,并且留有 API 扩展空间,使得这个框架非常先进和健壮。

Robot Framework 完全用 Python 开发,对于验收测试非常有用。该框架可以运行在 Java 和.NET 环境,同时支持跨平台,如 Windows、MacOS 和 Linux。它无疑是最易用的自动化测试框架,能允许开发者进行并行测试。

RedwoodHQ

这是一个流行的自动化测试工具,它之所以流行是因为它支持大部分流行的编程语言,如 Java、Python、C# 等。它还支持多个测试人员在一个平台上协作并运行测试用例。

RedwoodHQ 有一个内置的 IDE(集成开发环境),可以在那里创建、修改以及运行测试用例。RedwoodHQ 是对用户最友好或对测试人员最友好的平台之一,它关注一个重大项目的全部测试过程。

Jasmine

这是一套 Javascript 行为驱动开发测试框架(BDD),不依赖于其他任何框架和 DOM,适用于任何使用 JavaScript 的地方。除了 JavaScript 之外,Jasmine 还被用于 Python 和 Ruby 自动化测试。

因此,使用 Jasmine 可以并行运行客户端测试用例和服务端测试用例。它是一个将客户端和服务端单元测试结合起来的完美的测试框架,而且被认为是测试领域的未来。

Pytest

如果项目比较小、复杂度比较低,Pytest 是最适合的自动化测试平台。大部分 Python 开发者用它来进行单元测试。它也具有 Robot Framework 所闻名的验收测试能力。

Pytest 最好的特性之一是,它提供了测试用例的详细失败信息,使开发者可以快速准确地改正问题。它还有各种可用插件来给现有测试技术和测试用例增加更多功能和多样性。