HomeMesosphereNo notifications. 4 unresolved issues.
  • Search
Differential D287

changelog for Marathon 1.4.0
ClosedAll Users
Actions

Authored by aquamatthias on Dec 7 2016, 12:15 PM.

Details

Reviewers
jdef
sascala
• jasongilanfarr
Commits
rMARATHON5a7c35bb6e48: changelog for Marathon 1.4.0
rMARATHON59d8b2c8f05e: changelog for Marathon 1.4.0
rMARATHONdcb7496df25f: changelog for Marathon 1.4.0
Test Plan

Read

Diff Detail

Repository
rMARATHON marathon
Lint
Automatic diff as part of commit; lint not applicable.
Unit
Automatic diff as part of commit; unit tests not applicable.
There are a very large number of changes, so older changes are hidden. Show Older Changes
aquamatthias retitled this revision from to changelog for Marathon 1.4.0.Dec 7 2016, 12:15 PM
aquamatthias updated this object.
aquamatthias edited the test plan for this revision. (Show Details)
aquamatthias added reviewers: sascala, jdef.
jdef requested changes to this revision.Dec 7 2016, 2:45 PM
jdef added inline comments.
changelog.md
49

s/task/tasks/

57

this sentence is confusing. pods were not added to app-definition.

63

s/By the time this functionality was introduced,/At the time that health checks were first added to Marathon/

64

s/are/were only/

68

s/with a big/to a large/

This revision now requires changes to proceed.Dec 7 2016, 2:45 PM
aquamatthias updated this revision to Diff 1134.Dec 7 2016, 3:58 PM
aquamatthias marked 5 inline comments as done.
  • Incorporated feedback from review.
aquamatthias added a comment.Dec 7 2016, 3:59 PM

@jdef thanks - addressed your points.

changelog.md
57

"Pods are implemented as a new primitive in Marathon."

• jasongilanfarr requested changes to this revision.Dec 7 2016, 10:49 PM
• jasongilanfarr added a reviewer: • jasongilanfarr.
• jasongilanfarr added a subscriber: • jasongilanfarr.
• jasongilanfarr added inline comments.
changelog.md
2

1.4.0 should still be listed as unreleased.

11

Deprecations should come together.

49

this should be 1.2 if I'm not mistaken.

78–79

Please hold on this, I'd like to write a far more extensive section here.

This revision now requires changes to proceed.Dec 7 2016, 10:49 PM
sascala added a comment.Dec 7 2016, 11:03 PM

Edits are inline

changelog.md
57

Suggest: "We created a new primitive, PodDefinition, as well as new API endpoints. Read more about to use pods in our Pods Documentation and the /v2/pods section of the REST API Reference"

59

Is the above sentence necessary?

60

"Mesos-based"

oxford comma" "HTTP, HTTPS, and TCP"

62

Suggest: "Health checks are an integral part of application monitoring and have been available in Marathon since version 0.7."

64

Suggest: "Prior to tha availability of Mesos-based health checks, health checks were only performed directly in Marathon. This has the following consequences:"

65

Marathon has to share the same network as.....?

68

Suggest: "Marathon health checks do not scale to large numbers of tasks."

70

s/do/perform

71

Mesos-based

72

Remove "Please".

75

Suggest: "We strongly recommend Mesos-based health checks over Marathon-based health checks."

76

"Marathon-based"

81

Suggest: "Until version 1.3, Marathon used a flat storage layout in ZooKeeper and encountered this limitation with large installations.

82

Suggest: "The latest version of Marathon uses a nested storage layout, which significantly increases the number of nodes that can be stored."

85

"In prior versions,"

86

Suggest: "This could lead to a node size larger than 1 MB, which could not be stored."

87

"The latest version of Marathon stores a group only with references in order to keep node size under 1 MB".

89

Suggest: "A migration inside Marathon automatically migrates the prior layout to the new one."

94

Suggest: "When this happens, there is limited knowledge of the status of the agent's tasks."

95

remove the comma after "tasks"

97

Suggest: "Starting with Mesos 1.1, those task are declared unreachable, not lost."

98

Suggest: "The scheduler that launched the tasks decides how to handle unreachable tasks."

100

Suggest: "Marathon uses this feature and adds an unreachableStrategy to the AppDefinition and PodDefinition, which allows you to define:

101

"how long Marathon should"

102

"how long Marathon should"

107

"Insights into the Launch Process - AKA: Why isn't my app starting?

110

There are situations when Marathon cannot fulfill a launch request, since there is no matching offer from Mesos.

111

Suggest: "It was very hard for users to understand why Marathon could not fulfill launch requests."

112

Suggest: "The latest version of Marathon gives insight into the launch process by analyzing all incoming offers and providing the user with statistics. This makes it easier to see why offers were rejected."

115

Remove "Please"

122

"failover"?

123

Suggest: "This can be cumbersome if you have long-running updates and a Marathon failover."

124

failover?

127

"as an HTTP"

131

"Deprecate Marathon-based Health Checks"

133

"command-based" "network-based"

134

Suggest: "Since those health check types are now also available in Marathon, the Marathon-based health checks are now deprecated."

135

Do not use health checks with the following protocols: HTTP, HTTPS, and TCP. Instead, use the Mesos equivalents: MESOS_HTTP, MESOS_HTTPS and MESOS_TCP.

140

s/2/two

144

s/everybody/everyone

150

Suggest: "All actions currently being performed in Marathon will be stopped; the state will not change.

151

"This can lead to an inconsistent state and is dangerous."

aquamatthias updated this revision to Diff 1154.Dec 8 2016, 3:25 PM
aquamatthias marked 47 inline comments as done.
  • Incorporated feedback from review.
aquamatthias added a comment.Dec 8 2016, 3:26 PM

@sascala thanks for the suggestions.

changelog.md
49

No. It is 1.1.

59

My intent: for people that do not care about pods, make clear that apps still work

78–79

Happy to see an improvement in a follow up.

jdef requested changes to this revision.Dec 8 2016, 3:51 PM
jdef added inline comments.
changelog.md
64

s/tha/the/

101

2 points:

  1. the unreachable prefix for these fields was just removed
  2. formal field names should be formatted properly (like this)
113

???

This revision now requires changes to proceed.Dec 8 2016, 3:51 PM
aquamatthias updated this revision to Diff 1157.Dec 8 2016, 4:18 PM
aquamatthias marked 3 inline comments as done.
  • Incorporated feedback from review.
changelog.md
64

directly taken from suzanne :)

113

removed a line by accident.

jdef accepted this revision.Dec 8 2016, 4:55 PM
Closed by commit rMARATHONdcb7496df25f: changelog for Marathon 1.4.0 (authored by aquamatthias). · Explain WhyDec 8 2016, 5:46 PM
This revision was automatically updated to reflect the committed changes.

Revision Contents

PathPackages
Mchangelog.md (118 lines)

Diff 1162

View Options

changelog.md

11
2## Changes from 1.3.0 to 1.4.0 (unreleased version)
jasongilanfarrUnsubmitted
Done

1.4.0 should still be listed as unreleased.

23
3## Changes from 1.3.x to 1.4.0 (unreleased version)
4
54### Breaking Changes
65
7* Removed the deprecated `marathon_store_timeout` command line parameter. It was deprecated since v0.12 and unused.
86
97#### Plugin API has changed
108In order to support the nature of pods, we had to change the plugin interfaces in a backward incompatible fashion.
119Plugin writers need to update plugins, in order to use this version
1210
1311#### Health reporting via the event stream
jasongilanfarrUnsubmitted
Done

Deprecations should come together.

1412Adding support for pods in Marathon required the internal representation of tasks to be migrated to instances. An instance represents the executor on the Mesos side, and contains a list of tasks. This change is reflected in various parts of the API, which now accordingly reports health status etc for instances, not for tasks.
1513Until v1.3.x, Marathon published `health_status_changed_event`s via the event stream. With the introduction of instances that can contain multiple tasks, Marathon moved away from that event in favor of `instance_health_changed_event`s.
1614In case you were consuming that event you have to adjust your tooling to consume the introduced event instead, e.g.
1715```javascript
1816{
1917 "instanceId": "some_app.marathon-49d976d3-9c6f-11e6-93cb-0242216b9f0d",
2018 "runSpecId": "/some/app",
2119 "healthy": true,
Show All 16 Lines
3836- `unhealthy_instance_kill_event` (in favor of the previous `unhealthy_task_kill_event`) provides both the instanceId of the instance that got killed, as well as the taskId designating the task that failed health checks.
3937- Health information as reported via the apps and tasks endpoint.
4038
4139
4240
4341### Overview
4442
4543#### Pods
46(TODO)
4744
45A pod is a collection of co-located and co-scheduled containers in a shared context.
46The containers of a pod share a network namespace and may share access to the same filesystem(s).
47Each pod instance’s containers are individually resource-isolated.
48
49[Mesos 1.1](http://mesos.apache.org/blog/mesos-1-1-0-released) adds support for launching a group of tasks (LAUNCH_GROUP).
jdefUnsubmitted
Done

s/task/tasks/

jasongilanfarrUnsubmitted
Done

this should be 1.2 if I'm not mistaken.

aquamatthiasAuthorUnsubmitted
Not Done

No. It is 1.1.

50A pod instance’s containers are launched via this Mesos primitive.
51Mesos provides the executor implementation that Marathon will use to run pod instances.
52
53We created a new primitive, PodDefinition, as well as new API endpoints.
54Read more about to use pods in our [Pods Documentation](https://mesosphere.github.io/marathon/docs/pods.html),
55and the `/v2/pods` section of the [REST API Reference](https://mesosphere.github.io/marathon/docs/generated/api.html)
56
57Pods are implemented as a new primitive in Marathon.
jdefUnsubmitted
Done

this sentence is confusing. pods were not added to app-definition.

aquamatthiasAuthorUnsubmitted
Done

"Pods are implemented as a new primitive in Marathon."

sascalaUnsubmitted
Done

Suggest: "We created a new primitive, PodDefinition, as well as new API endpoints. Read more about to use pods in our Pods Documentation and the /v2/pods section of the REST API Reference"

58The general functionality of apps plus the related endpoints are still available.
59
sascalaUnsubmitted
Done

Is the above sentence necessary?

aquamatthiasAuthorUnsubmitted
Not Done

My intent: for people that do not care about pods, make clear that apps still work

60#### Mesos-based health checks for HTTP, HTTPS, and TCP
sascalaUnsubmitted
Done

"Mesos-based"

oxford comma" "HTTP, HTTPS, and TCP"

61
62Health checks are an integral part of application monitoring and have been available in Marathon since version 0.7.
sascalaUnsubmitted
Done

Suggest: "Health checks are an integral part of application monitoring and have been available in Marathon since version 0.7."

63At the time that health checks were first added to Marathon, there was no support for health checks in Mesos.
jdefUnsubmitted
Done

s/By the time this functionality was introduced,/At the time that health checks were first added to Marathon/

64Prior to the availability of Mesos-based health checks, health checks were only performed directly in Marathon. This has the following consequences:
jdefUnsubmitted
Done

s/are/were only/

sascalaUnsubmitted
Done

Suggest: "Prior to tha availability of Mesos-based health checks, health checks were only performed directly in Marathon. This has the following consequences:"

jdefUnsubmitted
Done

s/tha/the/

aquamatthiasAuthorUnsubmitted
Not Done

directly taken from suzanne :)

65- Marathon has to share the same network as the tasks to monitor, so it can reach all launched tasks
sascalaUnsubmitted
Done

Marathon has to share the same network as.....?

66- Network partitions could lead to wrong scheduling decisions
67- The health state is not available via the Mesos state
68- Marathon health checks do not scale to large numbers of tasks.
jdefUnsubmitted
Done

s/with a big/to a large/

sascalaUnsubmitted
Done

Suggest: "Marathon health checks do not scale to large numbers of tasks."

69
70Starting with Mesos 1.1, it is now possible to perform network based health checks directly on the Mesos executor level.
sascalaUnsubmitted
Done

s/do/perform

71Marathon makes all the Mesos-based health checks available.
sascalaUnsubmitted
Done

Mesos-based

72See the updated [Health Check Documentation](https://mesosphere.github.io/marathon/docs/health-checks.html),
sascalaUnsubmitted
Done

Remove "Please".

73especially the new protocols: `MESOS_HTTP`, `MESOS_HTTPS`, `MESOS_TCP`.
74
75We strongly recommend Mesos-based health checks over Marathon-based health checks.
sascalaUnsubmitted
Done

Suggest: "We strongly recommend Mesos-based health checks over Marathon-based health checks."

76Marathon-based health checks are deprecated and will be removed in a future version.
sascalaUnsubmitted
Done

"Marathon-based"

77
4878#### New ZK persistent storage layout
49(TODO)
5079
jasongilanfarrUnsubmitted
Done

Please hold on this, I'd like to write a far more extensive section here.

aquamatthiasAuthorUnsubmitted
Not Done

Happy to see an improvement in a follow up.

51### Fixed issues
52(TODO)
80ZooKeeper has a limitation on the number of nodes it can store in a directory node.
81Until version 1.3, Marathon used a flat storage layout in ZooKeeper and encountered this limitation with large installations.
sascalaUnsubmitted
Done

Suggest: "Until version 1.3, Marathon used a flat storage layout in ZooKeeper and encountered this limitation with large installations.

82The latest version of Marathon uses a nested storage layout, which significantly increases the number of nodes that can be stored.
sascalaUnsubmitted
Done

Suggest: "The latest version of Marathon uses a nested storage layout, which significantly increases the number of nodes that can be stored."

83
84ZooKeeper has a limitation on the size of one node (typically 1MB).
85In prior versions, a group was stored with all subgroups and applications.
sascalaUnsubmitted
Done

"In prior versions,"

86This could lead to a node size larger than 1 MB, which could not be stored.
sascalaUnsubmitted
Done

Suggest: "This could lead to a node size larger than 1 MB, which could not be stored."

87The latest version of Marathon stores a group only with references in order to keep node size under 1 MB.
sascalaUnsubmitted
Done

"The latest version of Marathon stores a group only with references in order to keep node size under 1 MB".

88
89A migration inside Marathon automatically migrates the prior layout to the new one.
sascalaUnsubmitted
Done

Suggest: "A migration inside Marathon automatically migrates the prior layout to the new one."

90
91#### Improve Task Lost behaviour
92
93The connection between the Mesos master and an agent can be broken for several reasons (network partition, agent update, etc).
94When this happens, there is limited knowledge of the status of the agent's tasks.
sascalaUnsubmitted
Done

Suggest: "When this happens, there is limited knowledge of the status of the agent's tasks."

95Prior versions of Mesos declared such tasks as lost after a timeout and killed the tasks if the agent rejoins the cluster.
sascalaUnsubmitted
Done

remove the comma after "tasks"

96
97Starting with Mesos 1.1, those task are declared unreachable, not lost.
sascalaUnsubmitted
Done

Suggest: "Starting with Mesos 1.1, those task are declared unreachable, not lost."

98The scheduler that launched the tasks decides how to handle unreachable tasks.
sascalaUnsubmitted
Done

Suggest: "The scheduler that launched the tasks decides how to handle unreachable tasks."

99
100Marathon uses this feature and adds an `unreachableStrategy` to the AppDefinition and PodDefinition, which allows you to define:
sascalaUnsubmitted
Done

Suggest: "Marathon uses this feature and adds an unreachableStrategy to the AppDefinition and PodDefinition, which allows you to define:

101- `inactiveAfterSeconds`: how long Marathon should wait to start a replacement task.
sascalaUnsubmitted
Done

"how long Marathon should"

jdefUnsubmitted
Done

2 points:

  1. the unreachable prefix for these fields was just removed
  2. formal field names should be formatted properly (like this)
102- `expungeAfterSeconds`: how long Marathon should wait for a task to come back.
sascalaUnsubmitted
Done

"how long Marathon should"

103
104If a task comes back and the replacement task is already started, Marathon needs to decide which task to kill.
105In order to let the user define which task should be taken, a kill selection can be defined.
106
107#### Insights into the Launch Process - AKA: Why isn't my app starting?
sascalaUnsubmitted
Done

"Insights into the Launch Process - AKA: Why isn't my app starting?

108
109Marathon tries to schedule tasks based on app or pod definition, which incorporates resource matching, role matching, constraint matching etc.
110There are situations when Marathon cannot fulfill a launch request, since there is no matching offer from Mesos.
sascalaUnsubmitted
Done

There are situations when Marathon cannot fulfill a launch request, since there is no matching offer from Mesos.

111It was very hard for users to understand why Marathon could not fulfill launch requests.
sascalaUnsubmitted
Done

Suggest: "It was very hard for users to understand why Marathon could not fulfill launch requests."

112For users that run into such situations, it was very hard to understand the reasons for this.
sascalaUnsubmitted
Done

Suggest: "The latest version of Marathon gives insight into the launch process by analyzing all incoming offers and providing the user with statistics. This makes it easier to see why offers were rejected."

113This version of Marathon gives insight into the launch process, analyzes all incoming offers and gives the user
jdefUnsubmitted
Done

???

aquamatthiasAuthorUnsubmitted
Not Done

removed a line by accident.

114statistics so it easy to see, why offers were rejected.
115
sascalaUnsubmitted
Done

Remove "Please"

116The statics can be fetched via the `/v2/queue` endpoint. See the [REST API Reference](https://mesosphere.github.io/marathon/docs/generated/api.html).
117Marathon shows the offer matching process as a funnel, so it easy to see how many offers were rejected in which step.
118It gives this information for the whole launch attempt as well as the last offer cycle.
119
120
121#### Improve Deployment logic
122
sascalaUnsubmitted
Done

"failover"?

123During Marathon master failover all deployments are started from the beginning.
sascalaUnsubmitted
Done

Suggest: "This can be cumbersome if you have long-running updates and a Marathon failover."

124This can be cumbersome if you have long-running updates and a Marathon failover.
sascalaUnsubmitted
Done

failover?

125This version of Marathon reconciles the state of a deployment after a failover.
126A running deployment will be continued on the new elected leader without restarting the deployment.
127
sascalaUnsubmitted
Done

"as an HTTP"

128Every state change operation via the REST API will now return the deployment identifier as an HTTP response header.
129
130### Deprecations
131
sascalaUnsubmitted
Done

"Deprecate Marathon-based Health Checks"

132#### Deprecate Marathon-based Health Checks
133
sascalaUnsubmitted
Done

"command-based" "network-based"

134Mesos now supports command-based as well as network-based health checks.
sascalaUnsubmitted
Done

Suggest: "Since those health check types are now also available in Marathon, the Marathon-based health checks are now deprecated."

135Since those health check types are now also available in Marathon, the Marathon-based health checks are now deprecated.
sascalaUnsubmitted
Done

Do not use health checks with the following protocols: HTTP, HTTPS, and TCP. Instead, use the Mesos equivalents: MESOS_HTTP, MESOS_HTTPS and MESOS_TCP.

136Do not use health checks with the following protocols: `HTTP`, `HTTPS`, and `TCP`. Instead, use the Mesos equivalents: `MESOS_HTTP`, `MESOS_HTTPS` and `MESOS_TCP`.
137
138
139#### Deprecate Event Callback Subscriptions
140
sascalaUnsubmitted
Done

s/2/two

141Marathon has two ways to subscribe to the internal event bus:
142- HTTP callback events managed via `/v2/eventSubscriptions`
143- Server Send Events via `/v2/events` (since Marathon 0.9)
144
sascalaUnsubmitted
Done

s/everybody/everyone

145We encourage everyone to use the `/v2/events` SSE stream instead of HTTP Callback listeners.
146The event callback subscriptions will be removed in a future version.
147
148#### Forcefully stop a deployment
149
150Deployments in Marathon can be stopped with force.
sascalaUnsubmitted
Done

Suggest: "All actions currently being performed in Marathon will be stopped; the state will not change.

151All actions currently being performed in Marathon will be stopped; the state will not change.
sascalaUnsubmitted
Done

"This can lead to an inconsistent state and is dangerous."

152This can lead to an inconsistent state and is dangerous.
153We will remove this functionality without replacement.
154
155#### Removed deprecated command line parameter
156Removed the deprecated `marathon_store_timeout` command line parameter. It was deprecated since v0.12 and unused.
53157
54158
55159## Changes from 1.3.5 to 1.3.6
56160
57161### Fixed issues:
58162
59163- When a runtime exit is requested, if the exit does not complete in less than the requested amount of time (10 seconds by default),
60164 now will actually kill the JVM. Previously, the timeout code did not actually work at all.
▲ Show 20 Lines • Show All 1599 Lines • Show Last 20 Lines