HomeMesosphereNo notifications. 4 unresolved issues.
  • Search
Differential D651

Tag all unstable tests again.
AbandonedAll Users
Actions

Authored by • jasongilanfarr on Mar 28 2017, 2:39 AM.

Details

Reviewers
jenkins
Commits
rMARATHON2abdad5bf080: Try a magical 'reload the jenkinsfile after applying the diff'
rMARATHONa12c591735cd: Files can only exist in nodes
rMARATHONdddf60ebec3f: Try to migrate the phabricator job into a library
rMARATHON2a93f26dff72: Groovy is a piece of f*ing s*it
rMARATHON54d29a97b60a: This is *soooo dumb*
rMARATHON7e6c498ba7ad: Jenkinsfile syntax is 'magical bs'
rMARATHONb02cff72e505: Switch to the ammonite version ( still a wip )
rMARATHONa226866c0726: Groovy is a piece of f*ing s*it
rMARATHON64b7858d696c: Try _yet another_ way to check the environment
rMARATHONc3aab4fcf9c5: Add assembly and package
rMARATHON65f88dc9ed91: Try env checking a different way
rMARATHON6b629859a115: Wrap in array
rMARATHON1c21714dd141: Tag all unstable tests again.
rMARATHONa19f00bbfa21: Jenkinsfile calling Jenkinsfile is _weird_
rMARATHON154e76f0c3c1: Cleanup jenkinsfile a lot and add unit test and coverage data
rMARATHONe9a267225c0d: Minor adjustments
rMARATHON738c1a4472ee: Now with more oomph
rMARATHON717d08e98350: This is getting ridiculous
JIRA Issues
JIRA MARATHON-7093 Fail build if after test, the git history is not clean.
JIRA MARATHON-7059 Jenkins <-> Phabricator Integration
JIRA MARATHON-2988 Build should check for formatting changes
JIRA MARATHON-7162 MesosAppIntegrationTest Fails on Unstable Master Loop
JIRA MARATHON-7161 RestartIntegrationTest Fails 5% of the Runs.
JIRA MARATHON-7160 TaskKillingIntegrationTest Fails with 404.
JIRA MARATHON-7159 AppDeployIntegrationTest and GroupDeployIntegrationTest Fail with 503 Timeout.
JIRA MARATHON-7158 ReadinessCheckIntegrationTest Fails 8 out of 100 Times.
JIRA MARATHON-7063 RestartIntegrationTest is flaky
JIRA MARATHON-7135 master builds and phab(diff)-triggered builds should use consistent test coverage rules
Summary
  • Tag all tests marked unstable in JIRA *with* the JIRA links
  • Run Unstable as part of the pipeline builds, but don't fail the overall build.
  • Create a phabricator-jenkins pipeline that does the same stuff, more or less (not done, can be split out).
    • NOTE: I already wrote this but it was deleted from jenkins!
Test Plan

Working on it.

Diff Detail

Repository
rMARATHON marathon
Branch
pipelines/jason/tag-unstable-again
Lint
No Linters Available
Unit
No Unit Test Coverage
Build Status
Buildable 2007
Build 3800: Marathon (deprecated)Jenkins
Build 3799: arc lint + arc unit
There are a very large number of changes, so older changes are hidden. Show Older Changes
• jasongilanfarr updated this revision to Diff 2486.Apr 4 2017, 12:43 AM
  • Groovy is a piece of f*ing s*it
jenkins requested changes to this revision.Apr 4 2017, 12:43 AM
This revision now requires changes to proceed.Apr 4 2017, 12:43 AM
• jasongilanfarr updated this revision to Diff 2503.Apr 4 2017, 10:13 PM
  • Try a magical 'reload the jenkinsfile after applying the diff'
• jasongilanfarr updated this revision to Diff 2504.Apr 4 2017, 10:16 PM
  • Jenkinsfile syntax is 'magical bs'
• jasongilanfarr updated this revision to Diff 2505.Apr 4 2017, 10:18 PM
  • This is getting ridiculous
jenkins requested changes to this revision.Apr 4 2017, 10:19 PM
This revision now requires changes to proceed.Apr 4 2017, 10:19 PM
• jasongilanfarr updated this revision to Diff 2506.Apr 4 2017, 10:21 PM
  • Files can only exist in nodes
jenkins requested changes to this revision.Apr 4 2017, 10:22 PM
This revision now requires changes to proceed.Apr 4 2017, 10:22 PM
• jasongilanfarr updated this revision to Diff 2507.Apr 4 2017, 10:32 PM
  • Minor adjustments
jenkins requested changes to this revision.Apr 4 2017, 10:33 PM
This revision now requires changes to proceed.Apr 4 2017, 10:33 PM
• jasongilanfarr updated this revision to Diff 2508.Apr 4 2017, 10:35 PM
  • This is *soooo dumb*
jenkins requested changes to this revision.Apr 4 2017, 10:36 PM
This revision now requires changes to proceed.Apr 4 2017, 10:36 PM
• jasongilanfarr updated this revision to Diff 2509.Apr 4 2017, 11:20 PM
  • Jenkinsfile calling Jenkinsfile is _weird_
jenkins requested changes to this revision.Apr 4 2017, 11:21 PM
This revision now requires changes to proceed.Apr 4 2017, 11:21 PM
• jasongilanfarr updated this revision to Diff 2510.Apr 4 2017, 11:25 PM
  • Try env checking a different way
jenkins requested changes to this revision.Apr 4 2017, 11:25 PM
This revision now requires changes to proceed.Apr 4 2017, 11:25 PM
• jasongilanfarr updated this revision to Diff 2511.Apr 4 2017, 11:28 PM
  • Try _yet another_ way to check the environment
jenkins requested changes to this revision.Apr 4 2017, 11:29 PM
This revision now requires changes to proceed.Apr 4 2017, 11:29 PM
• jasongilanfarr updated this revision to Diff 2514.Apr 4 2017, 11:53 PM
  • Jenkins can bite me
  • I hate jenkins
jenkins requested changes to this revision.Apr 4 2017, 11:53 PM
This revision now requires changes to proceed.Apr 4 2017, 11:53 PM
• jasongilanfarr updated this revision to Diff 2515.Apr 5 2017, 12:08 AM
  • I hate jenkins more and more.
jenkins requested changes to this revision.Apr 5 2017, 12:09 AM
This revision now requires changes to proceed.Apr 5 2017, 12:09 AM
• jasongilanfarr removed reviewers: timcharper, aquamatthias, jeschkies.Apr 5 2017, 12:40 AM
• jasongilanfarr removed subscribers: marathon-team, marathon-dev.

I'm using this one to test this out. please ignore.

• jasongilanfarr updated this revision to Diff 2518.Apr 5 2017, 12:53 AM
  • Fix silly issue
jenkins requested changes to this revision.Apr 5 2017, 12:53 AM
This revision now requires changes to proceed.Apr 5 2017, 12:53 AM

Revision Contents

PathPackages
MJenkinsfile (294 lines)
AMJenkinsfile.phab.groovy (130 lines)
AMbin/convert_test_coverage.scala (35 lines)
Mbuild.sbt (3 lines)
AMmarathon.groovy (248 lines)
Mtests/pre-checkin.sh (3 lines)

Diff 2518

View Options

Jenkinsfile

11#!/usr/bin/env groovy
2// so we can use marathon.groovy
3def m
24
3/**
4 * Execute block and set GitHub commit status to success or failure if block
5 * throws an exception.
6 *
7 * @param label The context for the commit status.
8 * @param block The block to execute.
9 */
10def withCommitStatus(label, block) {
11 try {
12 // Execute steps in stage
13 block()
145
15 currentBuild.result = 'SUCCESS'
16 } catch(error) {
17 currentBuild.result = 'FAILURE'
18 throw error
19 } finally {
20
21 // Mark commit with final status
22 step([ $class: 'GitHubCommitStatusSetter'
23 , contextSource: [$class: 'ManuallyEnteredCommitContextSource', context: "Velocity " + label]
24 ])
25 }
26}
27
28def previousBuildFailed() {
29 def previousResult = currentBuild.rawBuild.getPreviousBuild()?.getResult()
30 return !hudson.model.Result.SUCCESS.equals(previousResult)
31}
32
33/**
34 * Wrap block with a stage and a GitHub commit status setter.
35 *
36 * @param label The label for the stage and commit status context.
37 * @param block The block to execute in stage.
38 */
39def stageWithCommitStatus(label, block) {
40 stage(label) { withCommitStatus(label, block) }
41}
42
436node('JenkinsMarathonCI-Debian8-2017-03-21') {
447 try {
458 stage("Checkout Repo") {
469 checkout scm
4710 gitCommit = sh(returnStdout: true, script: 'git rev-parse HEAD').trim()
4811 shortCommit = gitCommit.take(8)
4912 currentBuild.displayName = "#${env.BUILD_NUMBER}: ${shortCommit}"
5013 }
14 m = load("marathon.groovy")
15
5116 stage("Kill junk processes") {
52 sh "bin/kill-stale-test-processes"
17 m.kill_junk()
5318 }
19 stage("Install Mesos") {
20 m.install_mesos()
21 }
5422 stageWithCommitStatus("1. Compile") {
5523 try {
56 sh """if grep -q MesosDebian \$WORKSPACE/project/Dependencies.scala; then
57 MESOS_VERSION=\$(sed -n 's/^.*MesosDebian = "\\(.*\\)"/\\1/p' <\$WORKSPACE/project/Dependencies.scala)
58 else
59 MESOS_VERSION=\$(sed -n 's/^.*mesos=\\(.*\\)&&.*/\\1/p' <\$WORKSPACE/Dockerfile)
60 fi
61 sudo apt-get install -y --force-yes --no-install-recommends mesos=\$MESOS_VERSION
62 """
63 withEnv(['RUN_DOCKER_INTEGRATION_TESTS=true', 'RUN_MESOS_INTEGRATION_TESTS=true']) {
64 sh "sudo -E sbt -Dsbt.log.format=false clean compile scapegoat doc"
65 }
24 m.compile()
6625 } finally {
6726 archiveArtifacts artifacts: 'target/**/scapegoat-report/scapegoat.html', allowEmptyArchive: true
6827 }
6928 }
7029 stageWithCommitStatus("2. Test") {
7130 try {
72 timeout(time: 30, unit: 'MINUTES') {
73 withEnv(['RUN_DOCKER_INTEGRATION_TESTS=true', 'RUN_MESOS_INTEGRATION_TESTS=true']) {
74 sh "sudo -E sbt -Dsbt.log.format=false coverage test coverageReport"
75 }
76 }
31 m.test()
7732 } finally {
78 junit allowEmptyResults: true, testResults: 'target/test-reports/**/*.xml'
7933 sh "sudo mv target/scala-2.11/scoverage-report/ target/scala-2.11/scoverage-report-unit"
8034 sh "sudo mv target/scala-2.11/coverage-report/cobertura.xml target/scala-2.11/coverage-report/cobertura-unit.xml"
8135 archiveArtifacts(
8236 artifacts: 'target/**/coverage-report/cobertura-unit.xml, target/**/scoverage-report-unit/**',
8337 allowEmptyArchive: true)
8438 }
8539 }
8640 stageWithCommitStatus("3. Test Integration") {
8741 try {
88 timeout(time: 30, unit: 'MINUTES') {
89 withEnv(['RUN_DOCKER_INTEGRATION_TESTS=true', 'RUN_MESOS_INTEGRATION_TESTS=true']) {
90 sh "sudo -E sbt -Dsbt.log.format=false clean coverage integration:test coverageReport mesos-simulation/integration:test"
91 }
92 }
42 m.integration_test()
9343 } finally {
94 junit allowEmptyResults: true, testResults: 'target/test-reports/*integration/**/*.xml'
9544 // scoverage does not allow the configuration of a different output
9645 // path: https://github.com/scoverage/sbt-scoverage/issues/211
9746 // The archive steps does not allow a different target path. So we
9847 // move the files to avoid conflicts with the reports from the unit
9948 // test run.
10049 sh "sudo mv target/scala-2.11/scoverage-report/ target/scala-2.11/scoverage-report-integration"
10150 sh "sudo mv target/scala-2.11/coverage-report/cobertura.xml target/scala-2.11/coverage-report/cobertura-integration.xml"
10251 archiveArtifacts(
10352 artifacts: 'target/**/coverage-report/cobertura-integration.xml, target/**/scoverage-report-integration/**',
10453 allowEmptyArchive: true)
10554 }
10655 }
10756 stage("4. Assemble Runnable Binaries") {
108 sh "sudo -E sbt assembly"
109 sh "sudo bin/build-distribution"
57 m.assembly()
11058 }
11159 stage("5. Package Binaries") {
112 parallel (
113 "Tar Binaries": {
114 sh """sudo tar -czv -f "target/marathon-${gitCommit}.tgz" \
115 Dockerfile \
116 README.md \
117 LICENSE \
118 bin \
119 examples \
120 docs \
121 target/scala-2.*/marathon-assembly-*.jar
122 """
123 },
124 "Create Debian and Red Hat Package": {
125 dir("packaging") {
126 sh "sudo make all"
60 m.package()
12761 }
128 },
129 "Build Docker Image": {
130 // target is in .dockerignore so we just copy the jar before.
131 sh "cp target/*/marathon-assembly-*.jar ."
132 mesosVersion = sh(returnStdout: true, script: "sed -n 's/^.*MesosDebian = \"\\(.*\\)\"/\\1/p' <./project/Dependencies.scala").trim()
133 docker.build("mesosphere/marathon:${gitCommit}", "--build-arg MESOS_VERSION=${mesosVersion} .")
62 stage("6. Run Unstable Tests") {
63 if (m.has_unstable_tests()) {
64 try {
65 m.unstable_test()
66 } catch (err) {
67 // For PRs, can we report it there somehow?
68 if (env.BRANCH_NAME.startsWith("releases/") || env.BRANCH_NAME == "master") {
69 slackSend(message: "\u26a0 branch `${env.BRANCH_NAME}` failed in build `${env.BUILD_NUMBER}`. (<${env.BUILD_URL}|Open>)",
70 color: "danger",
71 channel: "#marathon-dev",
72 tokenCredentialId: "f430eaac-958a-44cb-802a-6a943323a6a8")
13473 }
135 )
13674 }
137 stage("6. Archive Artifacts") {
75 }
76 }
77 stage("7. Archive Artifacts") {
13878 archiveArtifacts artifacts: 'target/**/classes/**', allowEmptyArchive: true
13979 archiveArtifacts artifacts: 'target/marathon-runnable.jar', allowEmptyArchive: true
14080 archiveArtifacts artifacts: "target/marathon-${gitCommit}.tgz", allowEmptyArchive: false
14181 archiveArtifacts artifacts: "packaging/marathon*.deb", allowEmptyArchive: false
14282 archiveArtifacts artifacts: "packaging/marathon*.rpm", allowEmptyArchive: false
jeschkiesUnsubmitted
Not Done

How are we going to run the unstable tests in a loop? Does this change imply that we get rid of the unstable job?

jasongilanfarrAuthorUnsubmitted
Not Done

This is a stop-gap to bring more transparency to unstable, we absolutely should be running things in a loop.

14383 step([
144 $class: 'S3BucketPublisher',
145 entries: [[
146 sourceFile: "target/marathon-${gitCommit}.tgz",
147 bucket: 'marathon-artifacts',
148 selectedRegion: 'us-west-2',
149 noUploadOnFailure: true,
150 managedArtifacts: true,
151 flatten: true,
84 $class : 'S3BucketPublisher',
85 entries : [[
86 sourceFile : "target/marathon-${gitCommit}.tgz",
87 bucket : 'marathon-artifacts',
88 selectedRegion : 'us-west-2',
89 noUploadOnFailure : true,
90 managedArtifacts : true,
91 flatten : true,
15292 showDirectlyInBrowser: false,
153 keepForever: true,
93 keepForever : true,
15494 ]],
155 profileName: 'marathon-artifacts',
95 profileName : 'marathon-artifacts',
15696 dontWaitForConcurrentBuildCompletion: false,
157 consoleLogLevel: 'INFO',
158 pluginFailureResultConstraint: 'FAILURE'
97 consoleLogLevel : 'INFO',
98 pluginFailureResultConstraint : 'FAILURE'
15999 ])
160100 }
161101 // Only create latest-dev snapshot for master.
162 if( env.BRANCH_NAME == "master" ) {
163 stage("7. Publish Docker Image Snaphot") {
102 if (env.BRANCH_NAME == "master") {
103 stage("8. Publish Docker Image Snaphot") {
164104 docker.image("mesosphere/marathon:${gitCommit}").tag("latest-dev")
165105 docker.withRegistry("https://index.docker.io/v1/", "docker-hub-credentials") {
166106 docker.image("mesosphere/marathon:latest-dev").push()
167107 }
168108 }
169109 }
170110 } catch (Exception err) {
171111 currentBuild.result = 'FAILURE'
172 if( env.BRANCH_NAME.startsWith("releases/") || env.BRANCH_NAME == "master" ) {
112 if (env.BRANCH_NAME.startsWith("releases/") || env.BRANCH_NAME == "master") {
173113 slackSend(
174114 message: "(;¬_¬) branch `${env.BRANCH_NAME}` failed in build `${env.BUILD_NUMBER}`. (<${env.BUILD_URL}|Open>)",
175115 color: "danger",
176116 channel: "#marathon-dev",
177117 tokenCredentialId: "f430eaac-958a-44cb-802a-6a943323a6a8")
178118 }
179119 throw err
180120 } finally {
181 if( env.BRANCH_NAME.startsWith("releases/") || env.BRANCH_NAME == "master" ) {
121 if (env.BRANCH_NAME.startsWith("releases/") || env.BRANCH_NAME == "master") {
182122 // Last build failed but this succeeded.
183 if( previousBuildFailed() && currentBuild.result == 'SUCCESS') {
123 if (previousBuildFailed() && currentBuild.result == 'SUCCESS') {
184124 slackSend(
185125 message: "╭( ･ㅂ･)و ̑̑ branch `${env.BRANCH_NAME}` is green again. (<${env.BUILD_URL}|Open>)",
186126 color: "good",
187127 channel: "#marathon-dev",
188128 tokenCredentialId: "f430eaac-958a-44cb-802a-6a943323a6a8")
189129 }
190130 }
191131
192 step([ $class: 'GitHubCommitStatusSetter'
132 step([$class : 'GitHubCommitStatusSetter'
193133 , errorHandlers: [[$class: 'ShallowAnyErrorHandler']]
194134 , contextSource: [$class: 'ManuallyEnteredCommitContextSource', context: "Velocity All"]
195135 ])
196136 }
197137}
View Options

Jenkinsfile.phab.groovy

  • This file was added.
1#!/usr/bin/env groovy
2
3
4/* BEGIN: Block of stuff that we can't have in the library for this job until the patch itself lands - chicken and egg: all in marathon.groovy */
5
6def setBuildInfo(displayName, description) {
7 currentBuild.displayName = displayName
8 currentBuild.description = description
9 return this
10}
11
12def install_dependencies() {
13 // JQ is broken in the image
14 sh "curl -L https://github.com/stedolan/jq/releases/download/jq-1.5/jq-linux64 > /tmp/jq && sudo mv /tmp/jq /usr/bin/jq && sudo chmod +x /usr/bin/jq"
15 // install ammonite (scala shell)
16 sh """mkdir -p ~/.ammonite && curl -L -o ~/.ammonite/predef.sc https://git.io/v6r5y && mkdir -p ~/.ammonite && curl -L -o ~/.ammonite/predef.sc https://git.io/v6r5y"""
17 sh """sudo curl -L -o /usr/local/bin/amm https://github.com/lihaoyi/Ammonite/releases/download/0.8.2/2.12-0.8.2 && sudo chmod +x /usr/local/bin/amm"""
18 return this
19}
20
21def checkout_marathon_master() {
22 git changelog: false, credentialsId: '4ff09dce-407b-41d3-847a-9e6609dd91b8', poll: false, url: 'git@github.com:mesosphere/marathon.git'
23 sh "sudo git clean -fdx"
24 return this
25}
26
27def phabricator_apply_diff(phid, build_url, revision_id, diff_id) {
28 phabricator("harbormaster.createartifact", """buildTargetPHID: "$phid", artifactType: "uri", artifactKey: "$build_url", artifactData: { uri: "$build_url", name: "Velocity Results", "ui.external": true }""")
29 phabricator("differential.revision.edit", """transactions: [{type: "reject", value: false}], objectIdentifier: "D$revision_id" """)
30 phabricator("harbormaster.sendmessage", """ buildTargetPHID: "$phid", type: "work" """)
31 sh "arc patch --diff $diff_id"
32}
33
34def phabricator(method, args) {
35 sh "jq -n '{ $args }' | arc call-conduit $method || true"
36 return this
37}
38
39def clean_git() {
40 sh "git checkout master && git branch | grep -v master | xargs git branch -D || true"
41 return this
42}
43
44// so we can use marathon.groovy later.
45def m
46/* END: Block of stuff that can be removed after the patch lands */
47
48ansiColor('gnome-terminal') {
49 node('JenkinsMarathonCI-Debian8-1-2017-03-21') {
50 if (fileExists('marathon.groovy')) {
51 m = load('marathon.groovy')
52 }
53 // once the diff containing this has been submitted, the if statement above can be
54 // removed and a lot of the function calls have to be prefixed with 'm'
55 setBuildInfo("D$REVISION_ID($DIFF_ID) #$BUILD_NUMBER", "<a href=\"https://phabricator.mesosphere.com/D$REVISION_ID\">D$REVISION_ID</a>")
56
57 stage("Install Dependencies") {
58 install_dependencies()
59 }
60 stage("Checkout D$REVISION_ID($DIFF_ID)") {
61 checkout_marathon_master()
62 clean_git()
63 phabricator_apply_diff("$PHID", "$BUILD_URL", "$REVISION_ID", "$DIFF_ID")
64 }
65
66 m = load('marathon.groovy')
67 if (m == null) {
68 error "Jenkins doesn't know how to load stuff."
69 }
70 try {
71 stage("Install Mesos") {
72 m.install_mesos()
73 }
74 stage("Kill junk processes") {
75 m.kill_junk()
76 }
77 stage("Compile") {
78 m.compile()
79 }
80 stage("Test") {
81 try {
82 m.test()
83 } finally {
84 m.phabricator_convert_test_coverage()
85 m.publish_test_coverage("Test Coverage")
86 }
87 }
88 stage("Integration Test") {
89 try {
90 m.integration_test()
91 } finally {
92 m.phabricator_convert_test_coverage()
93 m.publish_test_coverage("Integration Test Coverage")
94 }
95 }
96 stage("Assemble Binaries") {
97 m.assembly()
98 }
99 stage("Package Binaries") {
100 m.package_binaries()
101 }
102 stage("Unstable Test") {
103 if (m.has_unstable_tests()) {
104 try {
105 m.unstable_test()
106 } catch (err) {
107 m.phabricator("differential.revision.edit", """ transactions: [{type: "comment", value: "\u26a0 Warning $BUILD_URL has failing unstable tests!"}], objectIdentifier: "D$REVISION_ID" """)
108 } finally {
109 m.phabricator_convert_test_coverage()
110 m.publish_test_coverage("Unstable Test Coverage")
111 }
112 } else {
113 echo "No Unstable Tests \u2713"
114 }
115 }
116 stage("Publish Results") {
117 m.clean_git()
118 m.phabricator_test_results("fail")
119 m.phabricator("differential.revision.edit", """ transactions: [{type: "accept", value: true}, {type: "comment", value: "Build Succeeded at $BUILD_URL"}], objectIdentifier: "D$REVISION_ID" """)
120 }
121 } catch (Exception err) {
122 stage("Publish Results") {
123 m.clean_git()
124 m.phabricator_test_results("fail")
125 m.phabricator("differential.revision.edit", """ transactions: [{type: "reject", value: true}, {type: "comment", value: "Build Failed at $BUILD_URL"}], objectIdentifier: "D$REVISION_ID" """)
126 m.currentBuild.result = "FAILURE"
127 }
128 }
129 }
130}
View Options

bin/convert_test_coverage.scala

  • This file was added.
PropertyOld ValueNew Value
File Modenull100755
1#!/usr/local/bin/amm
2
3import $ivy.`com.typesafe.play:play-json_2.11:2.5.12`
4
5import scala.xml.XML
6import play.api.libs.json._
7
8val xml = XML.loadFile("target/scala-2.11/coverage-report/cobertura.xml")
9// Map(filename -> Map(lineNo -> hit count))
10val lineHits = (xml \\ "class").map { `class` =>
11 var results = Map.empty[Int, Int]
12 (`class` \\ "methods" \ "method" \ "lines" \ "line").map { line =>
13 val lineNo = (line \ "@number").text.toInt
14 val hits = results.get(lineNo).fold(0)(_ + (line \ "@hits").text.toInt)
15 results = results + (lineNo -> hits)
16 }
17
18 (`class` \ "@filename").text -> results
19}
20
21// N = Not Executable, C = Covered, U = NotCovered
22// Map(filename -> NNCCU)
23val coverage: Map[String, String] = lineHits.map { case (file, lineData) =>
24 file -> 1.to(lineData.keys.max).map { lineNo =>
25 lineData.get(lineNo) match {
26 case None => "N"
27 case Some(x) if x == 0 => "U"
28 case Some(x) => "C"
29 }
30 }.mkString("")
31}(collection.breakOut)
32
33case class HMTest(coverage: Map[String, String], result: String = "pass", name: String = "Test Coverage")
34implicit val hmtestFormat = Json.format[HMTest]
35println(Json.toJson(Seq(HMTest(coverage))))
View Options

build.sbt

Show First 20 Lines • Show All 146 Lines • ▼ Show 20 Line(s)
147147 testForkedParallel in IntegrationTest := true,
148148 concurrentRestrictions in IntegrationTest := Seq(Tags.limitAll(math.max(1, java.lang.Runtime.getRuntime.availableProcessors() / 2))),
149149 test in IntegrationTest := {
150150 (test in IntegrationTest).value
151151 (test in SerialIntegrationTest).value
152152 },
153153
154154 fork in UnstableIntegrationTest := true,
155155 testOptions in UnstableIntegrationTest := Seq(formattingTestArg(target.value / "test-reports" / "unstable-integration"),
156156 Tests.Argument(
157157 "-n", "mesosphere.marathon.IntegrationTest",
158 "-l", "mesosphere.marathon.SerialIntegrationTest",
158 "-n", "mesosphere.marathon.SerialIntegrationTest",
jeschkiesUnsubmitted
Not Done

Does this mean the serial tests are actually run in parallel as well?

jasongilanfarrAuthorUnsubmitted
Not Done

Yes otherwise we'd need UnstableSerialIntegration as well. I don't object to adding that _too_.

159159 "-y", "org.scalatest.WordSpec")),
160160 parallelExecution in UnstableIntegrationTest := true,
161 testForkedParallel in UnstableIntegrationTest := true,
161162
jeschkiesUnsubmitted
Not Done

If I'm not mistaken the change misses the tag.

162163 scapegoatVersion := "1.3.0",
163164
164165 coverageMinimum := 62,
165166 coverageFailOnMinimum := true,
166167
167168 fork in run := true,
168169 AspectjKeys.aspectjVersion in Aspectj := "1.8.10",
169170 AspectjKeys.inputs in Aspectj += compiledClasses.value,
▲ Show 20 Lines • Show All 139 Lines • Show Last 20 Lines
View Options

marathon.groovy

  • This file was added.
1// Libraries of methods for testing Marathon in jenkins
2
3// Add a prefix to all of the JUnit result files listed
4// This is particularly useful for tagging things like "Unstable-${TestName}"
5def setJUnitPrefix(prefix, files) {
6 // add prefix to qualified classname
7 sh """bash -c "shopt -s globstar && sed -i \"s/\\(<testcase .*classname=['\\\"]\\)\\([a-z]\\)/\\1${
8 prefix.toUpperCase()
9 }.\\2/g\" $files" """
10 return this
11}
12
13// Run the given phabricator method (e.g. arc call-conduit <method>) with
14// the given jq arguments wrapped in a json object.
15// e.g. phabricator("differential.revision.edit", """ transactions: [{type: "comment", "value": "Some comment"}], objectIdentifier: "D1" """)
16def phabricator(method, args) {
17 sh "jq -n '{ $args }' | arc call-conduit $method || true"
18 return this
19}
20
21// Report all the test results for the given PHID with the given status to Harbormaster.
22// PHID is expected to be set as an environment variable
23def phabricator_test_results(status) {
24 sh """jq -s add '{buildTargetPHID: "$PHID", type: "$status", unit: [.[] * .[]] }' target/phabricator-test-reports/* | arc call-conduit harbormaster.sendmessage """
25 return this
26}
27
28// Convert the test coverage into a "fake" unit test result so that
29// phabricator_test_results can consume it and report the coverage.
30def phabricator_convert_test_coverage() {
31 sh """sudo sh -c "/usr/local/bin/amm bin/convert_test_coverage.scala" """
32 return this
33}
34
35// Publish the test coverage information into the build.
36// Currently, none of the methods for reporting this actually work
37def publish_test_coverage(name) {
38 //currentBuild.description += "<h3>$name</h3>"
39 //currentBuild.description += readFile("target/scala-2.11/scoverage-report/index.html")
40 //publishHtml([allowMissing: true, alwaysLinkToLastBuild: false, keepAll: true, reportDir: 'target/scoverage-report', reportFiles: 'index.html', reportName: "Test Coverage"])
41 return this
42}
43
44// Applies the phabricator diff and posts messages to phabricator
45// that the build is in progress, the revision is rejected and
46// the harbormaster build has the given URL.
47// Ephid: the harbormaster phid to update.
48// build_url: The build URL of the jenkins build
49// revision_id: the revision id being built, e.g. D123
50// diff_id: The diff id to apply (e.g. 2458)
51def phabricator_apply_diff(phid, build_url, revision_id, diff_id) {
52 phabricator("harbormaster.createartifact", """buildTargetPHID: "$phid", artifactType: "uri", artifactKey: "$build_url", artifactData: { uri: "$build_url", name: "Velocity Results", "ui.external": true }""")
53 phabricator("differential.revision.edit", """transactions: [{type: "request-review", value: true}], objectIdentifier: "D$revision_id" """)
54 phabricator("harbormaster.sendmessage", """ buildTargetPHID: "$phid", type: "work" """)
55 sh "arc patch --diff $diff_id"
56}
57
58// installs mesos at the revision listed in the build.
59def install_mesos() {
60 sh """if grep -q MesosDebian \$WORKSPACE/project/Dependencies.scala; then
61 MESOS_VERSION=\$(sed -n 's/^.*MesosDebian = "\\(.*\\)"/\\1/p' <\$WORKSPACE/project/Dependencies.scala)
62 else
63 MESOS_VERSION=\$(sed -n 's/^.*mesos=\\(.*\\)&&.*/\\1/p' <\$WORKSPACE/Dockerfile)
64 fi
65 sudo apt-get install -y --force-yes --no-install-recommends mesos=\$MESOS_VERSION
66 """
67 return this
68}
69
70// Kill stale processes left-over from old builds.
71def kill_junk() {
72 sh "bin/kill-stale-test-processes"
73}
74
75// Install job-level dependencies that aren't specific to the build and
76// can be required as part of checkout and should be applied before knowing
77// the revision's information. e.g. JQ is required to post to phabricator.
78def install_dependencies() {
79 // JQ is broken in the image
80 sh "curl -L https://github.com/stedolan/jq/releases/download/jq-1.5/jq-linux64 > /tmp/jq && sudo mv /tmp/jq /usr/bin/jq && sudo chmod +x /usr/bin/jq"
81 // install ammonite (scala shell)
82 sh """mkdir -p ~/.ammonite && curl -L -o ~/.ammonite/predef.sc https://git.io/v6r5y && mkdir -p ~/.ammonite && curl -L -o ~/.ammonite/predef.sc https://git.io/v6r5y"""
83 sh """sudo curl -L -o /usr/local/bin/amm https://github.com/lihaoyi/Ammonite/releases/download/0.8.2/2.12-0.8.2 && sudo chmod +x /usr/local/bin/amm"""
84 return this
85}
86
87def setBuildInfo(displayName, description) {
88 currentBuild.displayName = displayName
89 currentBuild.description = description
90 return this
91}
92
93def checkout_marathon_master() {
94 git changelog: false, credentialsId: '4ff09dce-407b-41d3-847a-9e6609dd91b8', poll: false, url: 'git@github.com:mesosphere/marathon.git'
95 sh "sudo git clean -fdx"
96 return this
97}
98
99def clean_git() {
100 sh "git checkout master && git branch | grep -v master | xargs git branch -D || true"
101 return this
102}
103
104// run through compile/lint/docs. Fail if there were format changes after this.
105def compile() {
106 withEnv(['RUN_DOCKER_INTEGRATION_TESTS=true', 'RUN_MESOS_INTEGRATION_TESTS=true']) {
107 sh "sudo -E sbt -Dsbt.log.format=false clean scapegoat doc coverage test:compile"
108 sh """if git diff --quiet; then echo 'No format issues detected'; else echo 'Patch has Format Issues'; exit 1; fi"""
109 }
110}
111
112def test() {
113 try {
114 timeout(time: 30, unit: 'MINUTES') {
115 withEnv(['RUN_DOCKER_INTEGRATION_TESTS=true', 'RUN_MESOS_INTEGRATION_TESTS=true']) {
116 STATUS = sh(script: """sudo -E sbt -Dsbt.log.format=false coverage test""", returnStatus: true)
117 sh """sudo -E sbt -Dsbt.log.format=false coverageReport"""
118 if (STATUS != 0) {
119 error "Tests Failed"
120 }
121 }
122 }
123 } finally {
124 junit allowEmptyResults: true, testResults: 'target/test-reports/**/*.xml'
125 }
126}
127
128def integration_test() {
129 try {
130 timeout(time: 30, unit: 'MINUTES') {
131 withEnv(['RUN_DOCKER_INTEGRATION_TESTS=true', 'RUN_MESOS_INTEGRATION_TESTS=true']) {
132 STATUS = sh(script: """sudo -E sbt -Dsbt.log.format=false '; clean; coverage; integration:test; mesos-simulation/integration:test' """, returnStatus: true)
133 sh """sudo -E sbt -Dsbt.log.format=false '; set coverageFailOnMinimum := false; coverageReport'"""
134 if (STATUS != 0) {
135 error "Integration Tests Failed"
136 }
137 }
138 }
139 } finally {
140 junit allowEmptyResults: true, testResults: 'target/test-reports/*integration/**/*.xml'
141 }
142}
143
144def has_unstable_tests() {
145 return sh(script: "git grep \"@UnstableTest\" | wc -l", returnStdout: true) != "0"
146}
147
148def unstable_test() {
149 try {
150 timeout(time: 60, unit: 'MINUTES') {
151 withEnv(['RUN_DOCKER_INTEGRATION_TESTS=true', 'RUN_MESOS_INTEGRATION_TESTS=true']) {
152 // ignore the status here.
153 STATUS = sh(script: "sudo -E sbt -Dsbt.log.format=false clean coverage unstable:test unstable-integration:test", returnStatus: true)
154 sh """sudo -E sbt -Dsbt.log.format=false coverageReport"""
155 if (STATUS != 0) {
156 throw new Exception("Unstable Tests Failed.")
157 }
158 }
159 }
160 } finally {
161 setJUnitPrefix("Unstable", "target/test-reports/unstable-integration/**/*.xml")
162 setJUnitPrefix("Unstable", "target/test-reports/unstable/**/*.xml")
163 junit allowEmptyResults: true, testResults: 'target/test-reports/unstable-integration/**/*.xml'
164 junit allowEmptyResults: true, testResults: 'target/test-reports/unstable/**/*.xml'
165 }
166}
167
168def assembly() {
169 sh "sudo -E sbt assembly"
170 sh "sudo bin/build-distribution"
171}
172
173def package_binaries() {
174 parallel(
175 "Tar Binaries": {
176 sh """sudo tar -czv -f "target/marathon-${gitCommit}.tgz" \
177 Dockerfile \
178 README.md \
179 LICENSE \
180 bin \
181 examples \
182 docs \
183 target/scala-2.*/marathon-assembly-*.jar
184 """
185 },
186 "Create Debian and Red Hat Package": {
187 sh "sudo rm -rf marathon-pkg && git clone https://github.com/mesosphere/marathon-pkg.git marathon-pkg"
188 dir("marathon-pkg") {
189 // marathon-pkg has marathon as a git module. We've already
190 // checked it out. So let's just symlink.
191 sh "sudo rm -rf marathon && ln -s ../ marathon"
192 sh "sudo make all"
193 }
194 },
195 "Build Docker Image": {
196 // target is in .dockerignore so we just copy the jar before.
197 sh "cp target/*/marathon-assembly-*.jar ."
198 mesosVersion = sh(returnStdout: true, script: "sed -n 's/^.*MesosDebian = \"\\(.*\\)\"/\\1/p' <./project/Dependencies.scala").trim()
199 docker.build("mesosphere/marathon:${gitCommit}", "--build-arg MESOS_VERSION=${mesosVersion} .")
200 }
201 )
202}
203
204
205/**
206 * Execute block and set GitHub commit status to success or failure if block
207 * throws an exception.
208 *
209 * @param label The context for the commit status.
210 * @param block The block to execute.
211 */
212def withCommitStatus(label, block) {
213 try {
214 // Execute steps in stage
215 block()
216
217 currentBuild.result = 'SUCCESS'
218 } catch(error) {
219 currentBuild.result = 'FAILURE'
220 throw error
221 } finally {
222
223 // Mark commit with final status
224 step([ $class: 'GitHubCommitStatusSetter'
225 , contextSource: [$class: 'ManuallyEnteredCommitContextSource', context: "Velocity " + label]
226 ])
227 }
228}
229
230def previousBuildFailed() {
231 def previousResult = currentBuild.rawBuild.getPreviousBuild()?.getResult()
232 return !hudson.model.Result.SUCCESS.equals(previousResult)
233}
234
235/**
236 * Wrap block with a stage and a GitHub commit status setter.
237 *
238 * @param label The label for the stage and commit status context.
239 * @param block The block to execute in stage.
240 */
241def stageWithCommitStatus(label, block) {
242 stage(label) { withCommitStatus(label, block) }
243}
244
245
246// !!Important Boilerplate!!
247// The external code must return it's contents as an object
248return this
View Options

tests/pre-checkin.sh

Show All 26 Lines
2727sudo rm -rf $WORKSPACE/project/target/*
2828sudo rm -rf $WORKSPACE/mesos-simulation/target/*
2929
3030export RUN_DOCKER_INTEGRATION_TESTS=true
3131export RUN_MESOS_INTEGRATION_TESTS=true
3232
3333# Integration tests using the Mesos containerizer have to be run as superuser.
3434sudo -E sbt -Dsbt.log.format=false clean coverage test coverageReport compile scapegoat doc integration:test mesos-simulation/integration:test
35sudo mv target/scala-2.11/scoverage-report target/scala-2.11/scoverage-report-stable
36sudo mv target/scala-2.11/coverage-report/cobertura.xml target/scala-2.11/cobertura-stable.xml
37sudo -E sbt -Dsbt.log.format=false clean coverage unstable:test unstable-integration:test coverageReport || true
These changes have failed to build.