HomeMesosphereNo notifications. 4 unresolved issues.
  • Search
Differential D672

Enable port-mapping for Mesos container in Marathon on multiple container networks
ClosedAll Users
Actions

Authored by timcharper on Apr 5 2017, 9:33 AM.

Details

Reviewers
aquamatthias
jdef
jenkins
Commits
rMARATHONfe63d01ccf8e: Enable port-mapping for Mesos container in Marathon on multiple container…
rMARATHONc33840fe1330: Enable port-mapping for Mesos container in Marathon on multiple container…
rMARATHON15e85e5f30c5: Enable port-mapping for Mesos container in Marathon on multiple container…
rMARATHON0b6d4d170c00: Enable port-mapping for Mesos container in Marathon on multiple container…
JIRA Issues
JIRA MARATHON_EE-859 Enable port-mapping for Mesos containers in Marathon on multiple container networks
Summary

This change adds support for port mappings with multiple container networks by adding the "networkName" field to portMapping. When multiple container networks are specified, this field is required for any portMapping whose hostPort is defined. If only one network is defined, it is optional.

Test Plan

create cluster with multiple container networks. launch container with app that binds on different networks.

Diff Detail

Repository
rMARATHON marathon
Lint
Automatic diff as part of commit; lint not applicable.
Unit
Automatic diff as part of commit; unit tests not applicable.
There are a very large number of changes, so older changes are hidden. Show Older Changes
This revision now requires changes to proceed.Apr 18 2017, 2:26 AM
jdef requested changes to this revision.Apr 18 2017, 3:43 AM

mostly requesting changes because I think we dropped support for port DI for apps w/ host networking

src/main/scala/mesosphere/mesos/PortDiscovery.scala
17

this will be difficult without taking into account servicePort and requirePorts. let's chat about how to move forward

28

where's the implicit reader object for PortDefinition?

89

I think we're getting away from using for constructs in favor of map/flatMap

120

this func lost support for portDefinitions? that's not right...

src/test/scala/mesosphere/UnitTest.scala
80

nit: can we have a better name for template .. like message?

src/test/scala/mesosphere/mesos/PortDiscoveryTest.scala
170

nice. can we have a test for app w/ host networking (that uses PortDefinition)? I suspect adding one (without changing any of the DI generation code) will fail because of the missing support for PortDefinition

timcharper updated this revision to Diff 2703.Apr 18 2017, 7:59 AM

fix host ports DI generation

timcharper updated this revision to Diff 2704.Apr 18 2017, 8:09 AM

tuple reader was the worst idea ever

jenkins requested changes to this revision.Apr 18 2017, 8:11 AM
This revision now requires changes to proceed.Apr 18 2017, 8:11 AM
timcharper updated this revision to Diff 2705.Apr 18 2017, 8:15 AM

some comments

timcharper updated this revision to Diff 2706.Apr 18 2017, 8:22 AM
timcharper marked 6 inline comments as done.

code comment

jenkins requested changes to this revision.Apr 18 2017, 8:23 AM
This revision now requires changes to proceed.Apr 18 2017, 8:23 AM
timcharper updated this revision to Diff 2707.Apr 18 2017, 8:24 AM
timcharper marked 3 inline comments as done.

update UnitTest containerViolation param name

src/main/scala/mesosphere/mesos/PortDiscovery.scala
28

added

89

we should update our contributing.md file, in which it explicitly says:

BAD:
xs.flatMap(_.flatMap(_.map(_.z))) // replace with a for comprehension
171

JIRA MARATHON-7241 Document network-scope

timcharper updated this revision to Diff 2708.Apr 18 2017, 8:26 AM
timcharper marked 2 inline comments as done.

another commentr

timcharper updated this revision to Diff 2709.Apr 18 2017, 8:27 AM
timcharper marked 2 inline comments as done.

...

timcharper marked 5 inline comments as done.Apr 18 2017, 8:28 AM
jenkins requested changes to this revision.Apr 18 2017, 8:28 AM
This revision now requires changes to proceed.Apr 18 2017, 8:28 AM
jdef added a comment.Apr 18 2017, 3:24 PM

changeset looks great. minor changes requested

docs/docs/networking.md
148

nice

src/main/scala/mesosphere/marathon/state/Container.scala
72

only requirements apply when hostPort is set (networkNames must be a single-item list) and there are multiple networks in use by the app/pod. there's not enough "context" in this class to add a require or assume assertion because we don't have the list of networks used by the app/pod. but cleaning up the comment a bit would be appreciated

90

I think we need the equivalent of this (uniqueProtocols) for the RAML ContainerPortMapping validator in AppValidation

src/main/scala/mesosphere/mesos/PortDiscovery.scala
89

agreed. JIRA?

157

nit: s not needed, no substitution

timcharper marked 7 inline comments as done.Apr 18 2017, 6:24 PM
timcharper added inline comments.
src/main/scala/mesosphere/marathon/state/Container.scala
72

I see what you're saying. Yes, the context is a single PortMapping here, and the requirement concern is wider.

90

I don't think we do.

In RAML, it's an enum. Only three values are allowed:

docs/docs/rest-api/public/api/v2/types/stringTypes.raml

  39 |   NetworkProtocol:
  40 |     type: string
  41 |     enum: [tcp, udp, "udp,tcp"]
  42 |     description: Protocol of the port (tcp, udp)
  43 |     default: tcp
  44 |   TaskLostBehavior:
src/main/scala/mesosphere/mesos/PortDiscovery.scala
89

JIRA MARATHON-7244 Update contributing.md to reflect our consensus on for comprehensions

89

great; earlier, we discussed only rejecting style changes that were documented. Does that apply in this case?

timcharper updated this revision to Diff 2719.Apr 18 2017, 6:25 PM
timcharper marked 2 inline comments as done.

rebase; integrate changes

jenkins requested changes to this revision.Apr 18 2017, 6:25 PM
This revision now requires changes to proceed.Apr 18 2017, 6:25 PM
jdef accepted this revision.Apr 18 2017, 8:49 PM
                  *     .--.
                       / /  `
      +               | |
             '         \ \__,
         *          +   '--'  *
             +   /\
+              .'  '.   *
       *      /======\      +
             ;:.  _   ;
             |:. (_)  |
             |:.  _   |
   +         |:. (_)  |          *
             ;:.      ;
           .' \:.    / `.
          / .-'':._.'`-. \
          |/    /||\    \|
    jgs _..--"""````"""--.._
  _.-'``                    ``'-._
-'                                '-
src/main/scala/mesosphere/marathon/state/Container.scala
90

you're right. thanks for keeping me honest :)

src/main/scala/mesosphere/mesos/PortDiscovery.scala
89

i won't reject for this

aquamatthias accepted this revision.Apr 19 2017, 2:26 PM

Thanks!
James already did a great job in reviewing this PR.
Nice tests as well.

timcharper updated this revision to Diff 2755.Apr 19 2017, 9:57 PM

disable retry test logic in attempt to get sane pass / fail behavior

jenkins requested changes to this revision.Apr 19 2017, 9:58 PM
This revision now requires changes to proceed.Apr 19 2017, 9:58 PM
timcharper updated this revision to Diff 2760.Apr 19 2017, 10:52 PM

bring back retry-on-failed

This revision is now accepted and ready to land.Apr 19 2017, 10:54 PM
jenkins requested changes to this revision.Apr 19 2017, 10:54 PM
This revision now requires changes to proceed.Apr 19 2017, 10:54 PM
Closed by commit rMARATHON15e85e5f30c5: Enable port-mapping for Mesos container in Marathon on multiple container… (authored by timcharper). · Explain WhyApr 19 2017, 11:04 PM
This revision was automatically updated to reflect the committed changes.

Revision Contents

PathPackages
Mdocs/docs/networking.md (45 lines)
Mdocs/docs/rest-api/public/api/v2/schema/AppDefinition.json (10 lines)
Mdocs/docs/rest-api/public/api/v2/types/appContainer.raml (9 lines)
Mdocs/docs/rest-api/public/api/v2/types/network.raml (10 lines)
Msrc/main/java/mesosphere/marathon/Protos.java (312 lines)
Msrc/main/proto/marathon.proto (1 line)
Msrc/main/scala/mesosphere/marathon/api/serialization/ContainerSerializer.scala (92 lines)
Msrc/main/scala/mesosphere/marathon/api/v2/validation/AppValidation.scala (39 lines)
Msrc/main/scala/mesosphere/marathon/api/v2/validation/PodsValidation.scala (77 lines)
Msrc/main/scala/mesosphere/marathon/core/pod/Network.scala (13 lines)
Msrc/main/scala/mesosphere/marathon/core/task/state/NetworkInfo.scala (2 lines)
Msrc/main/scala/mesosphere/marathon/raml/NetworkConversion.scala (9 lines)
Msrc/main/scala/mesosphere/marathon/state/AppDefinition.scala (4 lines)
Msrc/main/scala/mesosphere/marathon/state/Container.scala (25 lines)
DMsrc/main/scala/mesosphere/mesos/NetworkScope.scala (22 lines)
Msrc/main/scala/mesosphere/mesos/PortDiscovery.scala (209 lines)
Msrc/main/scala/mesosphere/mesos/TaskBuilder.scala (2 lines)
Msrc/main/scala/mesosphere/mesos/TaskGroupBuilder.scala (86 lines)
Msrc/test/scala/mesosphere/UnitTest.scala (27 lines)
AMsrc/test/scala/mesosphere/marathon/api/serialization/ContainerSerializerTest.scala (53 lines)
Msrc/test/scala/mesosphere/marathon/api/v2/AppNormalizationTest.scala (21 lines)
Msrc/test/scala/mesosphere/marathon/api/v2/json/AppDefinitionSchemaJSONTest.scala (62 lines)
Msrc/test/scala/mesosphere/marathon/api/v2/json/AppDefinitionTest.scala (147 lines)
Msrc/test/scala/mesosphere/marathon/api/v2/validation/AppValidationTest.scala (148 lines)
Msrc/test/scala/mesosphere/marathon/api/v2/validation/PodsValidationTest.scala (159 lines)
Msrc/test/scala/mesosphere/marathon/raml/NetworkConversionTest.scala (38 lines)
Msrc/test/scala/mesosphere/marathon/state/AppDefinitionTest.scala (24 lines)
DMsrc/test/scala/mesosphere/marathon/state/PortMappingTest.scala (47 lines)
AMsrc/test/scala/mesosphere/mesos/PortDiscoveryTest.scala (226 lines)
Msrc/test/scala/mesosphere/mesos/TaskBuilderTest.scala (4 lines)
Msrc/test/scala/mesosphere/mesos/TaskGroupBuilderTest.scala (33 lines)

Diff 2761

View Options

docs/docs/networking.md

Show All 15 Lines
1616### Configuration
1717
1818Several [command-line flags](command-line-flags.html) determine Marathon's behavior with respect to networking.
1919
2020* `default_network_name` is injected as the `name` of a `container` mode network when left blank by an application.
2121* `local_port_min` and `local_port_max` define a port range from which Marathon automatically allocates *service-port*s.
2222
2323### Environment
24
2425* `MIGRATION_1_5_0_MARATHON_DEFAULT_NETWORK_NAME` is used when upgrading Marathon from a version prior to v1.5.
2526 * older MESOS IP/CT app definitions were not required to declare an `ipAddress/networkName`; Marathon 1.5 requires a resolvable network name.
2627 * migration automatically configures `container` networking mode for each migrated legacy MESOS IP/CT app.
2728 * at migration time, legacy MESOS IP/CT app definitions are configured to use the network name defined by the migration-specific environment variable above.
2829 * if the environment variable is **unset**, migration uses the network name defined by the `--default_network_name` flag.
2930 * if both the environment variable and flag are **unset**, app migration will fail.
jdefUnsubmitted
Done

bad change. please undo. ipAddress (and its fields) did not change for this PR

timcharperAuthorUnsubmitted
Not Done

Yep... thanks for catching; I was pressed for time with my presentation and didn't look through the patch carefully prior to putting up a revision

3031
3132## Networking Modes
3233
3334Marathon apps and pods declare `networks` the same way.
3435Three modes of networking are supported:
3536
3637### `host`:
38
3739An application will share the network namespace of the Mesos agent process, typically the host network namespace.
3840
3941### `container`:
42
4043An application should be allocated its own network namespace and IP address;
4144Mesos network isolators are responsible for providing backend support for this.
4245When using the Docker containerizer, this translates to a Docker "user" network.
4346Container networks are named, either explicitly by an application or else via `--default_network_name`.
4447Unnamed container networks will fail to validate if `--default_network_name` is not specified by the operator.
4548
4649### `container/bridge`:
50
4751Similar to `container`, an application should be allocated its own network namespace and IP address;
4852Mesos CNI provides a special `mesos-bridge` that application containers are attached to.
4953When using the Docker containerizer, this translates to the Docker "default bridge" network.
5054
5155*NOTE*: All network modes are supported for both the Mesos and Docker containerizers.
5256
53Usage:
57### Usage:
5458
55* An application may join one or more `container` mode networks. If joining multiple networks, `hostPort` may not be specified in `portMapping` declarations.
59* An application may join one or more `container` mode networks. When joining multiple container networks, additional restrictions are imposed on *port-mapping* entries (see *Port Mappings* for details).
jdefUnsubmitted
Done

s/specify/specifies/

jdefUnsubmitted
Done

nice!

jdefUnsubmitted
Not Done

When I re-read this description "each container network must be explicitly named" stuck me funny -- is that telling me that I'm not allowed to leverage the default network name feature for apps in app.networks? or that I can't specify a null list item entry in portMappings[x].networkNames in an attempt to pull in the "default" network for the mapping? (TBH I've never tested submitting arrays to Marathon with null entries)

Maybe I'm being a bit pedantic. I'm going to try "clarifying" things a bit:

  • An application may join one or more container mode networks. When joining multiple networks additional restrictions are imposed on *port-mapping* entries (see *port-mapping* for details).

And then in the *port-mapping* section:

When used in conjunction with multiple container mode networks, each mapping entry that specifies a non-null hostPort must declare a single item networkNames list that identifies the container mode network for which the mapping applies. null entries in networkNames are always ignored, for all cases.

timcharperAuthorUnsubmitted
Not Done

Good suggestion.

I took it a little further; have a look. (added note about network-name, fixed a statement made untrue by this patch, and made a note about the cartesian expansion effect of associated container networks and multiple protocols)

5660* An application may only join one `host` mode network; this is the default if an app definition does not declare a `networks` field.
5761* An application may only join one `container/bridge` network.
5862* An application cannot mix networking modes: either specify a single `host` network, or; single `container/bridge` network, or; one or more `container` networks.
5963
6064## Ports
6165
6266Marathon apps declare ports differently than pods.
6367The following section is relevant only for Marathon apps.
6468
6569### Terminology
6670
6771#### Port Types
6872
6973##### *container-port*:
74
7075Specifies a port within a container.
7176`containerPort` is specified as a field of a *port-mapping* or *endpoint* when using `container` or `container/bridge` mode networking.
7277
7378##### *host-port*:
79
7480Specifies a port to allocate from the resources offered by a Mesos agent.
7581`hostPort` is specified as a field of a *port-mapping* or *endpoint* when using `container` or `container/bridge` mode networking.
7682`port` is specified as a field of a *port-definition* when using `host` mode networking.
7783
7884**Note:** Only host ports are made available to a task through environment variables.
7985
8086##### *service-port*:
87
8188When you create a new application in Marathon (either through the REST API or the front end), you may assign one or more service ports to it.
8289You can specify all valid port numbers as service ports or you can use `0` to indicate that Marathon should allocate free service ports to the app automatically.
8390Marathon allocates service ports from the range defined by the `--local_port_min` and `--local_port_max` command line flags.
8491If you do choose your own service port, you have to ensure yourself that it is unique across all of your applications.
8592See [#port-definition].
8693Pods (endpoints) do not support service ports.
8794
8895#### Declaring ports in an application
8996
9097##### *endpoint*:
98
9199Endpoints are declared only by the containers of a Pod.
92100See the documentation for [pods](pods.html).
93101
94102##### *port-definition*:
103
95104Port-definitions are used only with `host` mode networking.
96105A *port-definition* (specifically its `port` field) is interpreted through the lens of the `requirePorts` app field:
97106
98107* When `requirePorts` is `false` (default), a port-definition's `port` is considered the *service-port* and a *host-port* is dynamically chosen by Marathon.
99108* When `requirePorts` is `true`, a port-definition's `port` is considered both a *host-port* and *service-port*, otherwise;
100109* The special `port` value of `0` tells Marathon to select any *host-port* from a Mesos resource offer and any *service-port* from the configured service port range.
101110
102111##### *port-mapping*:
112
103113A *port-mapping* declares a *container-port* for an application, possibly linking that *container-port* to a *host-port* and *service-port*.
104114Marathon communicates *container-port*/*host-port* links (aka "mappings") to Mesos when launching instances of the application.
105115Port-mappings are used with both `container` and `container/bridge` networking.
106116Marathon ignores the value of `requirePorts` when interpreting a *port-mapping*.
107117* The special `containerPort` value of `0` tells Marathon to internally assign the (eventually) allocated *host-port* to `containerPort`.
108118* The special `hostPort` value of `0` tells Marathon to select any *host-port* from a Mesos resource offer.
109119* The special `servicePort` value of `0` tells Marathon to select any *service-port* from the configured *service-port* range.
110120
111121### Port Definitions
112122
113123#### Summary
124
114125* Review *port-definition*, *host-port*, and *service-port* in [#Terminology].
115126* Location in app definition: `{ "portDefinitions": [ <port-definition>... ], "requirePorts": <bool>, ... }`
116127* Used in conjunction with `host` mode networking.
117128* `requirePorts` applies to `portDefinitions`.
118129* If no `portDefinitions` are defined (or defined as `null`) at create-time, default to `{ "portDefinitions": [ { "port": 0, "name": "default" } ], ... }`
119130 * Specify an empty array (`[]`) to indicate NO ports are used by the app; no default is injected in this case.
120131* Ignored when used in conjunction with other networking modes.
121132 * NOTE: Future versions of Marathon may fail to validate apps that declare `portDefinitions` with network modes other than `host`.
122133
123134### Port Mappings
124135
125136Summary:
137
126138* Review *port-mapping*, *container-port*, and *host-port* in [#Terminology].
127139* Location in app definition: `{ "container": { "portMappings": [ <port-mapping>... ], ... }, ... }`
128140* Used in conjunction with `container` and `container/bridge` mode networking.
129141* When using `container/bridge` mode networking an unspecified (`null`) value for `hostPort` is translated to `"hostPort": 0`.
130142* `requirePorts` does not apply to `portMappings`.
131143* If unspecified (`null`) at create-time, defaults to `{ "portMappings": [ { "containerPort": 0, "name": "default" } ], ... }`
132144 * Specify an empty array (`[]`) to indicate NO ports are used by the app; no default is injected in this case.
133145 * **NOTE:** When using `container/bridge` mode, the default *port-mapping* also sets `"hostPort: 0"`.
134146* Ignored when used in conjunction with other networking modes.
135147 * **NOTE:** Future versions of Marathon may fail to validate apps that declare `container.portMappings` with network modes other than `container` or `container/bridge`.
136* `hostPort` must be `null` when multiple `container` networks are declared by an application.
148* When used in conjunction with multiple container networks, each mapping entry that specifies a `hostPort` must also declare a `networkNames` value with a single item, identifying the network for which the mapping applies (a single `hostPort` may be mapped to only one container network, and `networkNames` defaults to all container networks for a pod or app).
jdefUnsubmitted
Done

nice

137 * **NOTE:** This restriction will likely be lifted in some future version of Marathon, once per-mapping network binding is exposed in the API.
138149
139150## Downward API
140151
141152### Per-Task Environment Variables
142153
143154If a port is named `NAME`, it will be accessible via the environment variable, `$PORT_NAME`.
144155Every *host-port* value is also exposed to the running application instance via environment variables `$PORT0`, `$PORT1`, etc.
145156Each Marathon application is given a single port by default, so `$PORT0` will normally be available, except for apps that specifically declare "no ports".
146157Variables are generated for all apps, regardless of whether the app uses the Mesos or Docker containerizer.
147158It is **highly recommended** to name the ports of an app to provide clarity with respect to the app configuration and intended use of each port.
148159
149160When using `container` or `container/bridge` mode networking, be sure to bind your application to the `containerPort`s you have specified in your `portMapping`s.
150161If you have set `containerPort` to `0`, this will be the same as `hostPort` and you can use the `$PORTxxx` environment variables.
151162
152163Additional [per-task enviroment variables](task-environment-variables.html) are also provided.
153164
154165### Discovery Via Mesos
155166
156167#### DiscoveryInfo and port `labels`:
157168
158169* `labels` may defined for items of `portDefinitions` as well as for items of `portMappings`. These labels are sent to Mesos via `DiscoveryInfo` protobufs at instance-launch time.
170* Given a mapping or endpoint, we generate a port `DiscoveryInfo` for every combination of specified protocols and associated networks. (IE, if an `Endpoint` specifies 2 protocols and is associated with 3 container networks, then a total of 6 `DiscoveryInfo` protobufs would be generated for that single `Endpoint`).
159171* Marathon injects a `network-scope` label into the port `DiscoveryInfo` to disambiguate between a *host-port* and *container-port*.
160172 * A scope value of `host` is used for *host-port* discovery.
161173 * A scope value of `container` is used for *container-port* discovery.
174* For *container-port* discovery, Marathon also injects a `network-name` label into the respective port `DiscoveryInfo`.
162175
163176#### Virtual addresses
164177
165178See the DC/OS documentation for [virtual addresses (VIPs)](https://dcos.io/docs/1.8/usage/service-discovery/virtual-ip-addresses/).
166179
167180## Examples
168181
169182### `host` Mode
▲ Show 20 Lines • Show All 251 Lines • Show Last 20 Lines
View Options

docs/docs/rest-api/public/api/v2/schema/AppDefinition.json

Show All 31 Lines
3232 "type": "string"
3333 }
3434 },
3535 "name": {
3636 "type": "string",
3737 "description": "Name of the service hosted on this port. If provided, it must be unique over all port mappings.",
3838 "pattern": "^[a-z][a-z0-9-]*$"
3939 },
40 "networkNames": {
41 "type": "array",
jdefUnsubmitted
Done

this should be an array of strings, not a string

42 "items": {
43 "type": "string",
44 "description": "List of the container networks associated with this mapping. If absent, then this mapping is associated with all defined container networks (for this application). A single item list is mandatory when `hostPort` is specified and multiple container networks are defined.",
45 "pattern": "^[a-z0-9]([-a-z0-9]*[a-z0-9])?$",
46 "minLength": 1,
47 "maxLength": 63
48 }
49 },
4050 "protocol": {
4151 "type": "string",
4252 "enum": [
4353 "tcp",
4454 "udp",
4555 "udp,tcp"
4656 ],
4757 "description": "Protocol of the port (one of ['tcp', 'udp'] or 'udp,tcp' for both). Defaults to 'tcp'."
▲ Show 20 Lines • Show All 767 Lines • Show Last 20 Lines
View Options

docs/docs/rest-api/public/api/v2/types/appContainer.raml

Show First 20 Lines • Show All 91 Lines • ▼ Show 20 Line(s)
9292 The servicePort parameter is optional and defaults to 0.
9393 Like hostPort, If the value is 0, a random port will be assigned.
9494 If a servicePort value is assigned by Marathon then Marathon
9595 guarantees that its value is unique across the cluster.
9696 The values for random service ports are in the range
9797 [local_port_min, local_port_max] where local_port_min and
9898 local_port_max are command line options with default values of
9999 10000 and 20000, respectively.
100 networkNames?:
jdefUnsubmitted
Done

(pragma.omitEmpty): for this field please

101 type: array
102 items: strings.Name
103 description: |
104 List of the container networks associated with this mapping. If
105 absent, then this mapping is associated with all defined container
106 networks (for this application). A single item list is mandatory when
107 `hostPort` is specified and multiple container networks are defined.
108 (pragma.omitEmpty):
100109 DockerContainer:
101110 type: object
102111 properties:
103112 credential?:
104113 type: DockerCredentials
105114 description: |
106115 The credentials to fetch this container.
107116 Please note: this property is supported only with the Mesos containerizer, not the docker containerizer.
▲ Show 20 Lines • Show All 70 Lines • Show Last 20 Lines
View Options

docs/docs/rest-api/public/api/v2/types/network.raml

Show All 35 Lines
3636 Specifying more than one protocol here will result in the generation
3737 of multiple Port DI entries.
3838 labels?:
3939 type: label.KVLabels
4040 (pragma.omitEmpty):
4141 description: |
4242 Metadata as key/value pair. Possible uses include VIPs,
4343 per-network-interface binding
44
44 networkNames?:
45 type: array
46 items: strings.Name
47 description: |
jdefUnsubmitted
Done

s/app/pod/

48 List of the container networks associated with this endpoint. If
49 absent, then this endpoint is associated with all defined container
50 networks (for this application). A single item list is mandatory when
51 `hostPort` is specified and multiple container networks are defined.
52 (pragma.omitEmpty):
jdefUnsubmitted
Done

(pragma.omitEmpty): for this field please

4553 NetworkMode:
4654 type: string
4755 enum: [container, "container/bridge", host]
4856 description: |
4957 `container` networks are named networks and are assigned an IP address by Mesos.
5058 The `container/bridge` network is a special container network that operates in a
5159 manner similar to Docker's "default bridge" mode, and depends on specific Mesos
5260 CNI support. The `host` network results in containers sharing the network namespace
▲ Show 20 Lines • Show All 94 Lines • Show Last 20 Lines
View Options

src/main/java/mesosphere/marathon/Protos.java

  • This file is larger than 256 KB, so syntax highlighting is disabled by default.
Show First 20 Lines • Show All 29134 Lines • ▼ Show 20 Line(s)
2913529135 /**
2913629136 * <code>optional uint32 service_port = 6 [default = 0];</code>
2913729137 */
2913829138 boolean hasServicePort();
2913929139 /**
2914029140 * <code>optional uint32 service_port = 6 [default = 0];</code>
2914129141 */
2914229142 int getServicePort();
29143
29144 // repeated string network_names = 7;
29145 /**
29146 * <code>repeated string network_names = 7;</code>
29147 */
29148 java.util.List<java.lang.String>
29149 getNetworkNamesList();
29150 /**
29151 * <code>repeated string network_names = 7;</code>
29152 */
29153 int getNetworkNamesCount();
29154 /**
29155 * <code>repeated string network_names = 7;</code>
29156 */
29157 java.lang.String getNetworkNames(int index);
29158 /**
29159 * <code>repeated string network_names = 7;</code>
29160 */
29161 com.google.protobuf.ByteString
29162 getNetworkNamesBytes(int index);
2914329163 }
2914429164 /**
2914529165 * Protobuf type {@code mesosphere.marathon.ExtendedContainerInfo.PortMapping}
2914629166 */
2914729167 public static final class PortMapping extends
2914829168 com.google.protobuf.GeneratedMessage
2914929169 implements PortMappingOrBuilder {
2915029170 // Use PortMapping.newBuilder() to construct.
▲ Show 20 Lines • Show All 69 Lines • ▼ Show 20 Line(s)
2922029240 labels_.add(input.readMessage(org.apache.mesos.Protos.Label.PARSER, extensionRegistry));
2922129241 break;
2922229242 }
2922329243 case 48: {
2922429244 bitField0_ |= 0x00000010;
2922529245 servicePort_ = input.readUInt32();
2922629246 break;
2922729247 }
29248 case 58: {
29249 if (!((mutable_bitField0_ & 0x00000040) == 0x00000040)) {
29250 networkNames_ = new com.google.protobuf.LazyStringArrayList();
29251 mutable_bitField0_ |= 0x00000040;
2922829252 }
29253 networkNames_.add(input.readBytes());
29254 break;
2922929255 }
29256 }
29257 }
2923029258 } catch (com.google.protobuf.InvalidProtocolBufferException e) {
2923129259 throw e.setUnfinishedMessage(this);
2923229260 } catch (java.io.IOException e) {
2923329261 throw new com.google.protobuf.InvalidProtocolBufferException(
2923429262 e.getMessage()).setUnfinishedMessage(this);
2923529263 } finally {
2923629264 if (((mutable_bitField0_ & 0x00000010) == 0x00000010)) {
2923729265 labels_ = java.util.Collections.unmodifiableList(labels_);
2923829266 }
29267 if (((mutable_bitField0_ & 0x00000040) == 0x00000040)) {
29268 networkNames_ = new com.google.protobuf.UnmodifiableLazyStringList(networkNames_);
29269 }
2923929270 this.unknownFields = unknownFields.build();
2924029271 makeExtensionsImmutable();
2924129272 }
2924229273 }
2924329274 public static final com.google.protobuf.Descriptors.Descriptor
2924429275 getDescriptor() {
2924529276 return mesosphere.marathon.Protos.internal_static_mesosphere_marathon_ExtendedContainerInfo_PortMapping_descriptor;
2924629277 }
▲ Show 20 Lines • Show All 186 Lines • ▼ Show 20 Line(s)
2943329464 }
2943429465 /**
2943529466 * <code>optional uint32 service_port = 6 [default = 0];</code>
2943629467 */
2943729468 public int getServicePort() {
2943829469 return servicePort_;
2943929470 }
2944029471
29472 // repeated string network_names = 7;
29473 public static final int NETWORK_NAMES_FIELD_NUMBER = 7;
29474 private com.google.protobuf.LazyStringList networkNames_;
29475 /**
29476 * <code>repeated string network_names = 7;</code>
29477 */
29478 public java.util.List<java.lang.String>
29479 getNetworkNamesList() {
29480 return networkNames_;
29481 }
29482 /**
29483 * <code>repeated string network_names = 7;</code>
29484 */
29485 public int getNetworkNamesCount() {
29486 return networkNames_.size();
29487 }
29488 /**
29489 * <code>repeated string network_names = 7;</code>
29490 */
29491 public java.lang.String getNetworkNames(int index) {
29492 return networkNames_.get(index);
29493 }
29494 /**
29495 * <code>repeated string network_names = 7;</code>
29496 */
29497 public com.google.protobuf.ByteString
29498 getNetworkNamesBytes(int index) {
29499 return networkNames_.getByteString(index);
29500 }
29501
2944129502 private void initFields() {
2944229503 hostPort_ = 0;
2944329504 containerPort_ = 0;
2944429505 protocol_ = "";
2944529506 name_ = "";
2944629507 labels_ = java.util.Collections.emptyList();
2944729508 servicePort_ = 0;
29509 networkNames_ = com.google.protobuf.LazyStringArrayList.EMPTY;
2944829510 }
2944929511 private byte memoizedIsInitialized = -1;
2945029512 public final boolean isInitialized() {
2945129513 byte isInitialized = memoizedIsInitialized;
2945229514 if (isInitialized != -1) return isInitialized == 1;
2945329515
2945429516 if (!hasContainerPort()) {
2945529517 memoizedIsInitialized = 0;
Show All 25 Lines
2948129543 output.writeBytes(4, getNameBytes());
2948229544 }
2948329545 for (int i = 0; i < labels_.size(); i++) {
2948429546 output.writeMessage(5, labels_.get(i));
2948529547 }
2948629548 if (((bitField0_ & 0x00000010) == 0x00000010)) {
2948729549 output.writeUInt32(6, servicePort_);
2948829550 }
29551 for (int i = 0; i < networkNames_.size(); i++) {
29552 output.writeBytes(7, networkNames_.getByteString(i));
29553 }
2948929554 getUnknownFields().writeTo(output);
2949029555 }
2949129556
2949229557 private int memoizedSerializedSize = -1;
2949329558 public int getSerializedSize() {
2949429559 int size = memoizedSerializedSize;
2949529560 if (size != -1) return size;
2949629561
Show All 17 Lines
2951429579 for (int i = 0; i < labels_.size(); i++) {
2951529580 size += com.google.protobuf.CodedOutputStream
2951629581 .computeMessageSize(5, labels_.get(i));
2951729582 }
2951829583 if (((bitField0_ & 0x00000010) == 0x00000010)) {
2951929584 size += com.google.protobuf.CodedOutputStream
2952029585 .computeUInt32Size(6, servicePort_);
2952129586 }
29587 {
29588 int dataSize = 0;
29589 for (int i = 0; i < networkNames_.size(); i++) {
29590 dataSize += com.google.protobuf.CodedOutputStream
29591 .computeBytesSizeNoTag(networkNames_.getByteString(i));
29592 }
29593 size += dataSize;
29594 size += 1 * getNetworkNamesList().size();
29595 }
2952229596 size += getUnknownFields().getSerializedSize();
2952329597 memoizedSerializedSize = size;
2952429598 return size;
2952529599 }
2952629600
2952729601 private static final long serialVersionUID = 0L;
2952829602 @java.lang.Override
2952929603 protected java.lang.Object writeReplace()
▲ Show 20 Lines • Show All 117 Lines • ▼ Show 20 Line(s)
2964729721 if (labelsBuilder_ == null) {
2964829722 labels_ = java.util.Collections.emptyList();
2964929723 bitField0_ = (bitField0_ & ~0x00000010);
2965029724 } else {
2965129725 labelsBuilder_.clear();
2965229726 }
2965329727 servicePort_ = 0;
2965429728 bitField0_ = (bitField0_ & ~0x00000020);
29729 networkNames_ = com.google.protobuf.LazyStringArrayList.EMPTY;
29730 bitField0_ = (bitField0_ & ~0x00000040);
2965529731 return this;
2965629732 }
2965729733
2965829734 public Builder clone() {
2965929735 return create().mergeFrom(buildPartial());
2966029736 }
2966129737
2966229738 public com.google.protobuf.Descriptors.Descriptor
▲ Show 20 Lines • Show All 41 Lines • ▼ Show 20 Line(s)
2970429780 result.labels_ = labels_;
2970529781 } else {
2970629782 result.labels_ = labelsBuilder_.build();
2970729783 }
2970829784 if (((from_bitField0_ & 0x00000020) == 0x00000020)) {
2970929785 to_bitField0_ |= 0x00000010;
2971029786 }
2971129787 result.servicePort_ = servicePort_;
29788 if (((bitField0_ & 0x00000040) == 0x00000040)) {
29789 networkNames_ = new com.google.protobuf.UnmodifiableLazyStringList(
29790 networkNames_);
29791 bitField0_ = (bitField0_ & ~0x00000040);
29792 }
29793 result.networkNames_ = networkNames_;
2971229794 result.bitField0_ = to_bitField0_;
2971329795 onBuilt();
2971429796 return result;
2971529797 }
2971629798
2971729799 public Builder mergeFrom(com.google.protobuf.Message other) {
2971829800 if (other instanceof mesosphere.marathon.Protos.ExtendedContainerInfo.PortMapping) {
2971929801 return mergeFrom((mesosphere.marathon.Protos.ExtendedContainerInfo.PortMapping)other);
▲ Show 20 Lines • Show All 45 Lines • ▼ Show 20 Line(s)
2976529847 } else {
2976629848 labelsBuilder_.addAllMessages(other.labels_);
2976729849 }
2976829850 }
2976929851 }
2977029852 if (other.hasServicePort()) {
2977129853 setServicePort(other.getServicePort());
2977229854 }
29855 if (!other.networkNames_.isEmpty()) {
29856 if (networkNames_.isEmpty()) {
29857 networkNames_ = other.networkNames_;
29858 bitField0_ = (bitField0_ & ~0x00000040);
29859 } else {
29860 ensureNetworkNamesIsMutable();
29861 networkNames_.addAll(other.networkNames_);
29862 }
29863 onChanged();
29864 }
2977329865 this.mergeUnknownFields(other.getUnknownFields());
2977429866 return this;
2977529867 }
2977629868
2977729869 public final boolean isInitialized() {
2977829870 if (!hasContainerPort()) {
2977929871
2978029872 return false;
▲ Show 20 Lines • Show All 508 Lines • ▼ Show 20 Line(s)
3028930381 */
3029030382 public Builder clearServicePort() {
3029130383 bitField0_ = (bitField0_ & ~0x00000020);
3029230384 servicePort_ = 0;
3029330385 onChanged();
3029430386 return this;
3029530387 }
3029630388
30389 // repeated string network_names = 7;
30390 private com.google.protobuf.LazyStringList networkNames_ = com.google.protobuf.LazyStringArrayList.EMPTY;
30391 private void ensureNetworkNamesIsMutable() {
30392 if (!((bitField0_ & 0x00000040) == 0x00000040)) {
30393 networkNames_ = new com.google.protobuf.LazyStringArrayList(networkNames_);
30394 bitField0_ |= 0x00000040;
30395 }
30396 }
30397 /**
30398 * <code>repeated string network_names = 7;</code>
30399 */
30400 public java.util.List<java.lang.String>
30401 getNetworkNamesList() {
30402 return java.util.Collections.unmodifiableList(networkNames_);
30403 }
30404 /**
30405 * <code>repeated string network_names = 7;</code>
30406 */
30407 public int getNetworkNamesCount() {
30408 return networkNames_.size();
30409 }
30410 /**
30411 * <code>repeated string network_names = 7;</code>
30412 */
30413 public java.lang.String getNetworkNames(int index) {
30414 return networkNames_.get(index);
30415 }
30416 /**
30417 * <code>repeated string network_names = 7;</code>
30418 */
30419 public com.google.protobuf.ByteString
30420 getNetworkNamesBytes(int index) {
30421 return networkNames_.getByteString(index);
30422 }
30423 /**
30424 * <code>repeated string network_names = 7;</code>
30425 */
30426 public Builder setNetworkNames(
30427 int index, java.lang.String value) {
30428 if (value == null) {
30429 throw new NullPointerException();
30430 }
30431 ensureNetworkNamesIsMutable();
30432 networkNames_.set(index, value);
30433 onChanged();
30434 return this;
30435 }
30436 /**
30437 * <code>repeated string network_names = 7;</code>
30438 */
30439 public Builder addNetworkNames(
30440 java.lang.String value) {
30441 if (value == null) {
30442 throw new NullPointerException();
30443 }
30444 ensureNetworkNamesIsMutable();
30445 networkNames_.add(value);
30446 onChanged();
30447 return this;
30448 }
30449 /**
30450 * <code>repeated string network_names = 7;</code>
30451 */
30452 public Builder addAllNetworkNames(
30453 java.lang.Iterable<java.lang.String> values) {
30454 ensureNetworkNamesIsMutable();
30455 super.addAll(values, networkNames_);
30456 onChanged();
30457 return this;
30458 }
30459 /**
30460 * <code>repeated string network_names = 7;</code>
30461 */
30462 public Builder clearNetworkNames() {
30463 networkNames_ = com.google.protobuf.LazyStringArrayList.EMPTY;
30464 bitField0_ = (bitField0_ & ~0x00000040);
30465 onChanged();
30466 return this;
30467 }
30468 /**
30469 * <code>repeated string network_names = 7;</code>
30470 */
30471 public Builder addNetworkNamesBytes(
30472 com.google.protobuf.ByteString value) {
30473 if (value == null) {
30474 throw new NullPointerException();
30475 }
30476 ensureNetworkNamesIsMutable();
30477 networkNames_.add(value);
30478 onChanged();
30479 return this;
30480 }
30481
3029730482 // @@protoc_insertion_point(builder_scope:mesosphere.marathon.ExtendedContainerInfo.PortMapping)
3029830483 }
3029930484
3030030485 static {
3030130486 defaultInstance = new PortMapping(true);
3030230487 defaultInstance.initFields();
3030330488 }
3030430489
▲ Show 20 Lines • Show All 17347 Lines • ▼ Show 20 Line(s)
4765247837 "own\020\005\"\330\001\n\tCondition\022\013\n\007Invalid\020\000\022\014\n\010Rese" +
4765347838 "rved\020\001\022\013\n\007Created\020\002\022\t\n\005Error\020\003\022\n\n\006Failed" +
4765447839 "\020\004\022\014\n\010Finished\020\005\022\n\n\006Killed\020\006\022\013\n\007Killing\020" +
4765547840 "\007\022\010\n\004Lost\020\010\022\013\n\007Running\020\t\022\013\n\007Staging\020\n\022\014\n" +
4765647841 "\010Starting\020\013\022\017\n\013Unreachable\020\014\022\010\n\004Gone\020\r\022\013" +
4765747842 "\n\007Unknown\020\016\022\013\n\007Dropped\020\017\"M\n\013MarathonApp\022" +
4765847843 "\014\n\004name\030\001 \001(\t\0220\n\005tasks\030\002 \003(\0132!.mesospher" +
4765947844 "e.marathon.MarathonTask\"1\n\rContainerInfo" +
47660 "\022\017\n\005image\030\001 \002(\014:\000\022\017\n\007options\030\002 \003(\014\"\316\t\n\025E",
47845 "\022\017\n\005image\030\001 \002(\014:\000\022\017\n\007options\030\002 \003(\014\"\345\t\n\025E",
4766147846 "xtendedContainerInfo\022\'\n\004type\030\001 \002(\0162\031.mes" +
4766247847 "os.ContainerInfo.Type\022,\n\007volumes\030\002 \003(\0132\033" +
4766347848 ".mesosphere.marathon.Volume\022M\n\rport_mapp" +
4766447849 "ings\030\006 \003(\01326.mesosphere.marathon.Extende" +
4766547850 "dContainerInfo.PortMapping\022E\n\006docker\030\003 \001" +
4766647851 "(\01325.mesosphere.marathon.ExtendedContain" +
4766747852 "erInfo.DockerInfo\022O\n\013mesosDocker\030\004 \001(\0132:" +
4766847853 ".mesosphere.marathon.ExtendedContainerIn" +
Show All 11 Lines
4768047865 "_port\030\001 \001(\r\022\026\n\016container_port\030\002 \002(\r\022\020\n\010p",
4768147866 "rotocol\030\003 \001(\t\022\014\n\004name\030\004 \001(\t\022\034\n\006labels\030\005 " +
4768247867 "\003(\0132\014.mesos.Label\022\027\n\014service_port\030d \001(\r:" +
4768347868 "\0010\032a\n\017MesosDockerInfo\022\r\n\005image\030\001 \002(\t\022%\n\n" +
4768447869 "credential\030\002 \001(\0132\021.mesos.Credential\022\030\n\020f" +
4768547870 "orce_pull_image\030\003 \001(\010\032b\n\rMesosAppCInfo\022\r" +
4768647871 "\n\005image\030\001 \002(\t\022\n\n\002id\030\002 \001(\t\022\034\n\006labels\030\003 \003(" +
4768747872 "\0132\014.mesos.Label\022\030\n\020force_pull_image\030\004 \001(" +
47688 "\010\032\217\001\n\013PortMapping\022\021\n\thost_port\030\001 \001(\r\022\026\n\016" +
47873 "\010\032\246\001\n\013PortMapping\022\021\n\thost_port\030\001 \001(\r\022\026\n\016" +
4768947874 "container_port\030\002 \002(\r\022\020\n\010protocol\030\003 \001(\t\022\014" +
4769047875 "\n\004name\030\004 \001(\t\022\034\n\006labels\030\005 \003(\0132\014.mesos.Lab",
47691 "el\022\027\n\014service_port\030\006 \001(\r:\0010\"\377\003\n\006Volume\022 " +
47692 "\n\004mode\030\003 \002(\0162\022.mesos.Volume.Mode\022\026\n\016cont" +
47693 "ainer_path\030\001 \002(\t\022\021\n\thost_path\030\002 \001(\t\022\033\n\005i" +
47694 "mage\030\004 \001(\0132\014.mesos.Image\022D\n\npersistent\030\005" +
47695 " \001(\01320.mesosphere.marathon.Volume.Persis" +
47696 "tentVolumeInfo\022@\n\010external\030\006 \001(\0132..mesos" +
47697 "phere.marathon.Volume.ExternalVolumeInfo" +
47698 "\032\237\001\n\024PersistentVolumeInfo\022\014\n\004size\030\001 \002(\004\022" +
47699 "2\n\004type\030\002 \001(\0162$.mesos.Resource.DiskInfo." +
47700 "Source.Type\0224\n\013constraints\030\003 \003(\0132\037.mesos",
47701 "phere.marathon.Constraint\022\017\n\007maxSize\030\004 \001" +
47702 "(\004\032a\n\022ExternalVolumeInfo\022\014\n\004size\030\001 \001(\004\022\014" +
47703 "\n\004name\030\002 \002(\t\022\020\n\010provider\030\003 \002(\t\022\035\n\007option" +
47704 "s\030\004 \003(\0132\014.mesos.Label\")\n\020EventSubscriber" +
47705 "s\022\025\n\rcallback_urls\030\001 \003(\t\"\274\001\n\016StorageVers" +
47706 "ion\022\r\n\005major\030\001 \002(\r\022\r\n\005minor\030\002 \002(\r\022\r\n\005pat" +
47707 "ch\030\003 \002(\r\022I\n\006format\030\004 \001(\01621.mesosphere.ma" +
47708 "rathon.StorageVersion.StorageFormat:\006LEG" +
47709 "ACY\"2\n\rStorageFormat\022\n\n\006LEGACY\020\000\022\025\n\021PERS" +
47710 "ISTENCE_STORE\020\001\"Z\n\031UpgradeStrategyDefini",
47711 "tion\022\035\n\025minimumHealthCapacity\030\001 \002(\001\022\036\n\023m" +
47712 "aximumOverCapacity\030\002 \001(\001:\0011\"\236\003\n\017GroupDef" +
47713 "inition\022\n\n\002id\030\001 \002(\t\022\017\n\007version\030\002 \002(\t\022?\n\017" +
47714 "deprecated_apps\030\003 \003(\0132&.mesosphere.marat" +
47715 "hon.ServiceDefinition\0222\n\017deprecated_pods" +
47716 "\030\010 \003(\0132\031.mesosphere.marathon.Json\0224\n\006gro" +
47717 "ups\030\004 \003(\0132$.mesosphere.marathon.GroupDef" +
47718 "inition\022\024\n\014dependencies\030\005 \003(\t\022?\n\004apps\030\006 " +
47719 "\003(\01321.mesosphere.marathon.GroupDefinitio" +
47720 "n.AppReference\022?\n\004pods\030\007 \003(\01321.mesospher",
47721 "e.marathon.GroupDefinition.AppReference\032" +
47722 "+\n\014AppReference\022\n\n\002id\030\001 \002(\t\022\017\n\007version\030\002" +
47723 " \002(\t\"\371\001\n\030DeploymentPlanDefinition\022\n\n\002id\030" +
47724 "\001 \002(\t\022\021\n\ttimestamp\030\002 \001(\t\022A\n\023deprecated_o" +
47725 "riginal\030\004 \001(\0132$.mesosphere.marathon.Grou" +
47726 "pDefinition\022?\n\021deprecated_target\030\005 \001(\0132$" +
47727 ".mesosphere.marathon.GroupDefinition\022\035\n\025" +
47728 "original_root_version\030\006 \001(\t\022\033\n\023target_ro" +
47729 "ot_version\030\007 \001(\t\"\306\001\n\013TaskFailure\022\016\n\006app_" +
47730 "id\030\001 \002(\t\022\036\n\007task_id\030\002 \002(\0132\r.mesos.TaskID",
47731 "\022\037\n\005state\030\003 \002(\0162\020.mesos.TaskState\022\021\n\007mes" +
47732 "sage\030\004 \001(\t:\000\022\016\n\004host\030\005 \001(\t:\000\022\017\n\007version\030" +
47733 "\006 \002(\t\022\021\n\ttimestamp\030\007 \002(\t\022\037\n\007slaveId\030\010 \001(" +
47734 "\0132\016.mesos.SlaveID\"T\n\014ZKStoreEntry\022\014\n\004nam" +
47735 "e\030\001 \002(\t\022\014\n\004uuid\030\002 \002(\014\022\r\n\005value\030\003 \002(\014\022\031\n\n" +
47736 "compressed\030\004 \001(\010:\005false\"\326\001\n\023ResidencyDef" +
47737 "inition\022(\n relaunchEscalationTimeoutSeco" +
47738 "nds\030\001 \001(\003\022S\n\020taskLostBehavior\030\002 \001(\01629.me" +
47739 "sosphere.marathon.ResidencyDefinition.Ta" +
47740 "skLostBehavior\"@\n\020TaskLostBehavior\022\032\n\026RE",
47741 "LAUNCH_AFTER_TIMEOUT\020\000\022\020\n\014WAIT_FOREVER\020\001" +
47742 "\"$\n\006Secret\022\n\n\002id\030\001 \002(\t\022\016\n\006source\030\002 \002(\t\"\262" +
47743 "\001\n\017EnvVarReference\0227\n\004type\030\001 \002(\0162).mesos" +
47744 "phere.marathon.EnvVarReference.Type\022\014\n\004n" +
47745 "ame\030\002 \002(\t\0227\n\tsecretRef\030\003 \001(\0132$.mesospher" +
47746 "e.marathon.EnvVarSecretRef\"\037\n\004Type\022\013\n\007UN" +
47747 "KNOWN\020\000\022\n\n\006SECRET\020\001\"#\n\017EnvVarSecretRef\022\020" +
47748 "\n\010secretId\030\001 \002(\t*3\n\rKillSelection\022\021\n\rYou" +
47749 "ngestFirst\020\001\022\017\n\013OldestFirst\020\002B\035\n\023mesosph" +
47750 "ere.marathonB\006Protos"
47876 "el\022\027\n\014service_port\030\006 \001(\r:\0010\022\025\n\rnetwork_n" +
47877 "ames\030\007 \003(\t\"\377\003\n\006Volume\022 \n\004mode\030\003 \002(\0162\022.me" +
47878 "sos.Volume.Mode\022\026\n\016container_path\030\001 \002(\t\022" +
47879 "\021\n\thost_path\030\002 \001(\t\022\033\n\005image\030\004 \001(\0132\014.meso" +
47880 "s.Image\022D\n\npersistent\030\005 \001(\01320.mesosphere" +
47881 ".marathon.Volume.PersistentVolumeInfo\022@\n" +
47882 "\010external\030\006 \001(\0132..mesosphere.marathon.Vo" +
47883 "lume.ExternalVolumeInfo\032\237\001\n\024PersistentVo" +
47884 "lumeInfo\022\014\n\004size\030\001 \002(\004\0222\n\004type\030\002 \001(\0162$.m" +
47885 "esos.Resource.DiskInfo.Source.Type\0224\n\013co",
47886 "nstraints\030\003 \003(\0132\037.mesosphere.marathon.Co" +
47887 "nstraint\022\017\n\007maxSize\030\004 \001(\004\032a\n\022ExternalVol" +
47888 "umeInfo\022\014\n\004size\030\001 \001(\004\022\014\n\004name\030\002 \002(\t\022\020\n\010p" +
47889 "rovider\030\003 \002(\t\022\035\n\007options\030\004 \003(\0132\014.mesos.L" +
47890 "abel\")\n\020EventSubscribers\022\025\n\rcallback_url" +
47891 "s\030\001 \003(\t\"\274\001\n\016StorageVersion\022\r\n\005major\030\001 \002(" +
47892 "\r\022\r\n\005minor\030\002 \002(\r\022\r\n\005patch\030\003 \002(\r\022I\n\006forma" +
47893 "t\030\004 \001(\01621.mesosphere.marathon.StorageVer" +
47894 "sion.StorageFormat:\006LEGACY\"2\n\rStorageFor" +
47895 "mat\022\n\n\006LEGACY\020\000\022\025\n\021PERSISTENCE_STORE\020\001\"Z",
47896 "\n\031UpgradeStrategyDefinition\022\035\n\025minimumHe" +
47897 "althCapacity\030\001 \002(\001\022\036\n\023maximumOverCapacit" +
47898 "y\030\002 \001(\001:\0011\"\236\003\n\017GroupDefinition\022\n\n\002id\030\001 \002" +
47899 "(\t\022\017\n\007version\030\002 \002(\t\022?\n\017deprecated_apps\030\003" +
47900 " \003(\0132&.mesosphere.marathon.ServiceDefini" +
47901 "tion\0222\n\017deprecated_pods\030\010 \003(\0132\031.mesosphe" +
47902 "re.marathon.Json\0224\n\006groups\030\004 \003(\0132$.mesos" +
47903 "phere.marathon.GroupDefinition\022\024\n\014depend" +
47904 "encies\030\005 \003(\t\022?\n\004apps\030\006 \003(\01321.mesosphere." +
47905 "marathon.GroupDefinition.AppReference\022?\n",
47906 "\004pods\030\007 \003(\01321.mesosphere.marathon.GroupD" +
47907 "efinition.AppReference\032+\n\014AppReference\022\n" +
47908 "\n\002id\030\001 \002(\t\022\017\n\007version\030\002 \002(\t\"\371\001\n\030Deployme" +
47909 "ntPlanDefinition\022\n\n\002id\030\001 \002(\t\022\021\n\ttimestam" +
47910 "p\030\002 \001(\t\022A\n\023deprecated_original\030\004 \001(\0132$.m" +
47911 "esosphere.marathon.GroupDefinition\022?\n\021de" +
47912 "precated_target\030\005 \001(\0132$.mesosphere.marat" +
47913 "hon.GroupDefinition\022\035\n\025original_root_ver" +
47914 "sion\030\006 \001(\t\022\033\n\023target_root_version\030\007 \001(\t\"" +
47915 "\306\001\n\013TaskFailure\022\016\n\006app_id\030\001 \002(\t\022\036\n\007task_",
47916 "id\030\002 \002(\0132\r.mesos.TaskID\022\037\n\005state\030\003 \002(\0162\020" +
47917 ".mesos.TaskState\022\021\n\007message\030\004 \001(\t:\000\022\016\n\004h" +
47918 "ost\030\005 \001(\t:\000\022\017\n\007version\030\006 \002(\t\022\021\n\ttimestam" +
47919 "p\030\007 \002(\t\022\037\n\007slaveId\030\010 \001(\0132\016.mesos.SlaveID" +
47920 "\"T\n\014ZKStoreEntry\022\014\n\004name\030\001 \002(\t\022\014\n\004uuid\030\002" +
47921 " \002(\014\022\r\n\005value\030\003 \002(\014\022\031\n\ncompressed\030\004 \001(\010:" +
47922 "\005false\"\326\001\n\023ResidencyDefinition\022(\n relaun" +
47923 "chEscalationTimeoutSeconds\030\001 \001(\003\022S\n\020task" +
47924 "LostBehavior\030\002 \001(\01629.mesosphere.marathon" +
47925 ".ResidencyDefinition.TaskLostBehavior\"@\n",
47926 "\020TaskLostBehavior\022\032\n\026RELAUNCH_AFTER_TIME" +
47927 "OUT\020\000\022\020\n\014WAIT_FOREVER\020\001\"$\n\006Secret\022\n\n\002id\030" +
47928 "\001 \002(\t\022\016\n\006source\030\002 \002(\t\"\262\001\n\017EnvVarReferenc" +
47929 "e\0227\n\004type\030\001 \002(\0162).mesosphere.marathon.En" +
47930 "vVarReference.Type\022\014\n\004name\030\002 \002(\t\0227\n\tsecr" +
47931 "etRef\030\003 \001(\0132$.mesosphere.marathon.EnvVar" +
47932 "SecretRef\"\037\n\004Type\022\013\n\007UNKNOWN\020\000\022\n\n\006SECRET" +
47933 "\020\001\"#\n\017EnvVarSecretRef\022\020\n\010secretId\030\001 \002(\t*" +
47934 "3\n\rKillSelection\022\021\n\rYoungestFirst\020\001\022\017\n\013O" +
47935 "ldestFirst\020\002B\035\n\023mesosphere.marathonB\006Pro",
47936 "tos"
4775147937 };
4775247938 com.google.protobuf.Descriptors.FileDescriptor.InternalDescriptorAssigner assigner =
4775347939 new com.google.protobuf.Descriptors.FileDescriptor.InternalDescriptorAssigner() {
4775447940 public com.google.protobuf.ExtensionRegistry assignDescriptors(
4775547941 com.google.protobuf.Descriptors.FileDescriptor root) {
4775647942 descriptor = root;
4775747943 internal_static_mesosphere_marathon_Constraint_descriptor =
4775847944 getDescriptor().getMessageTypes().get(0);
▲ Show 20 Lines • Show All 121 Lines • ▼ Show 20 Line(s)
4788048066 com.google.protobuf.GeneratedMessage.FieldAccessorTable(
4788148067 internal_static_mesosphere_marathon_ExtendedContainerInfo_MesosAppCInfo_descriptor,
4788248068 new java.lang.String[] { "Image", "Id", "Labels", "ForcePullImage", });
4788348069 internal_static_mesosphere_marathon_ExtendedContainerInfo_PortMapping_descriptor =
4788448070 internal_static_mesosphere_marathon_ExtendedContainerInfo_descriptor.getNestedTypes().get(3);
4788548071 internal_static_mesosphere_marathon_ExtendedContainerInfo_PortMapping_fieldAccessorTable = new
4788648072 com.google.protobuf.GeneratedMessage.FieldAccessorTable(
4788748073 internal_static_mesosphere_marathon_ExtendedContainerInfo_PortMapping_descriptor,
47888 new java.lang.String[] { "HostPort", "ContainerPort", "Protocol", "Name", "Labels", "ServicePort", });
48074 new java.lang.String[] { "HostPort", "ContainerPort", "Protocol", "Name", "Labels", "ServicePort", "NetworkNames", });
4788948075 internal_static_mesosphere_marathon_Volume_descriptor =
4789048076 getDescriptor().getMessageTypes().get(14);
4789148077 internal_static_mesosphere_marathon_Volume_fieldAccessorTable = new
4789248078 com.google.protobuf.GeneratedMessage.FieldAccessorTable(
4789348079 internal_static_mesosphere_marathon_Volume_descriptor,
4789448080 new java.lang.String[] { "Mode", "ContainerPath", "HostPath", "Image", "Persistent", "External", });
4789548081 internal_static_mesosphere_marathon_Volume_PersistentVolumeInfo_descriptor =
4789648082 internal_static_mesosphere_marathon_Volume_descriptor.getNestedTypes().get(0);
▲ Show 20 Lines • Show All 94 Lines • Show Last 20 Lines
View Options

src/main/proto/marathon.proto

Show First 20 Lines • Show All 304 Lines • ▼ Show 20 Line(s)
305305
306306 message PortMapping {
307307 optional uint32 host_port = 1;
308308 required uint32 container_port = 2;
309309 optional string protocol = 3;
310310 optional string name = 4;
311311 repeated mesos.Label labels = 5;
312312 optional uint32 service_port = 6 [default = 0];
313 repeated string network_names = 7;
313314 }
314315
315316 required mesos.ContainerInfo.Type type = 1;
316317 repeated Volume volumes = 2;
317318 repeated PortMapping port_mappings = 6; // since 1.5
318319
319320 optional DockerInfo docker = 3;
320321 optional MesosDockerInfo mesosDocker = 4;
▲ Show 20 Lines • Show All 155 Lines • Show Last 20 Lines
View Options

src/main/scala/mesosphere/marathon/api/serialization/ContainerSerializer.scala

11package mesosphere.marathon
22package api.serialization
33
44import mesosphere.marathon.core.externalvolume.ExternalVolumes
55import mesosphere.marathon.core.pod.{ BridgeNetwork, ContainerNetwork, HostNetwork, Network }
6import mesosphere.marathon.raml.Endpoint
76import mesosphere.marathon.state.Container.PortMapping
87import mesosphere.marathon.state._
98import mesosphere.marathon.stream.Implicits._
109import mesosphere.mesos.protos.Implicits._
1110import org.apache.mesos
1211
1312object ContainerSerializer {
1413 def fromProto(proto: Protos.ExtendedContainerInfo): Container = {
Show All 30 Lines
4544 builder.setType(mesos.Protos.ContainerInfo.Type.MESOS)
4645 builder.setMesosAppC(MesosAppCSerializer.toProto(ma))
4746 }
4847
4948 builder.build
5049 }
5150
5251 def toMesos(networks: Seq[Network], container: Container, mesosBridgeName: String): mesos.Protos.ContainerInfo = {
52
53 def portMappingToMesos(mapping: PortMapping) = {
54 val portBuilder = mesos.Protos.NetworkInfo.PortMapping.newBuilder()
jdefUnsubmitted
Done

remove comment

55 .setContainerPort(mapping.containerPort)
56 .setProtocol(mapping.protocol)
57 mapping.hostPort.foreach(portBuilder.setHostPort)
58 portBuilder.build
59 }
60
5361 val builder = mesos.Protos.ContainerInfo.newBuilder
5462
5563 // First set type-specific values (for Docker) because the external volume provider
5664 // might depend on it to set further values (builder is passed down as arg below).
5765 container match {
5866 case _: Container.Mesos =>
5967 builder.setType(mesos.Protos.ContainerInfo.Type.MESOS)
6068 case dd: Container.Docker =>
6169 builder.setType(mesos.Protos.ContainerInfo.Type.DOCKER)
6270 builder.setDocker(DockerSerializer.toMesos(dd))
6371 case md: Container.MesosDocker =>
6472 builder.setType(mesos.Protos.ContainerInfo.Type.MESOS)
6573 builder.setMesos(MesosDockerSerializer.toMesos(md))
6674 case ma: Container.MesosAppC =>
6775 builder.setType(mesos.Protos.ContainerInfo.Type.MESOS)
6876 builder.setMesos(MesosAppCSerializer.toMesos(ma))
6977 }
7078
7179 container.volumes.foreach {
7280 case _: PersistentVolume => // PersistentVolumes are handled differently
7381 case ev: ExternalVolume => ExternalVolumes.build(builder, ev) // this also adds the volume
7482 case dv: DockerVolume => builder.addVolumes(VolumeSerializer.toMesos(dv))
7583 }
7684
77 val networkInfos = networks.withFilter(_ != HostNetwork).map { network =>
85 networks.toIterator
86 .filter(_ != HostNetwork)
87 .map { network =>
7888 val (networkName, networkLabels) = network match {
7989 case cnet: ContainerNetwork => cnet.name -> cnet.labels.toMesosLabels
8090 case bnet: BridgeNetwork => mesosBridgeName -> bnet.labels.toMesosLabels
8191 case unsupported => throw new IllegalStateException(s"unsupported networking mode $unsupported")
8292 }
8393
94 val portMappings = container.portMappings.withFilter(_.hostPort.nonEmpty).withFilter { mapping =>
95 // if hostPort is specified, a SINGLE networkName is required.
96 // If it is empty then we've already validated that there is only one container network
jdefUnsubmitted
Done

"we assume there is only one network"?

97 mapping.networkNames.forall(_ == networkName)
98 }
99
84100 mesos.Protos.NetworkInfo.newBuilder()
85101 .addIpAddresses(mesos.Protos.NetworkInfo.IPAddress.getDefaultInstance)
86102 .setLabels(networkLabels)
87103 .setName(networkName)
88 .addAllPortMappings(container.portMappings.withFilter(_.hostPort.nonEmpty).map { mapping =>
89 // for now, all port mappings are bound to all non-host networks; this will likely change in the future.
90 val portBuilder = mesos.Protos.NetworkInfo.PortMapping.newBuilder()
91 .setContainerPort(mapping.containerPort)
92 .setProtocol(mapping.protocol)
93 mapping.hostPort.foreach(portBuilder.setHostPort)
94 portBuilder.build
95 })
104 .addAllPortMappings(portMappings.map(portMappingToMesos))
96105 .build
97106 }
98 builder.addAllNetworkInfos(networkInfos)
107 .foreach(builder.addNetworkInfos)
108
99109 builder.build
100110 }
101111}
102112
103113object VolumeSerializer {
104114 def toProto(volume: Volume): Protos.Volume = volume match {
105115 case p: PersistentVolume =>
106116 Protos.Volume.newBuilder()
▲ Show 20 Lines • Show All 104 Lines • ▼ Show 20 Line(s)
211221 val builder = Protos.ExtendedContainerInfo.PortMapping.newBuilder
212222 .setContainerPort(mapping.containerPort)
213223 .setProtocol(mapping.protocol)
214224 .setServicePort(mapping.servicePort)
215225
216226 mapping.hostPort.foreach(builder.setHostPort)
217227 mapping.name.foreach(builder.setName)
218228 mapping.labels.toProto.foreach(builder.addLabels)
229 mapping.networkNames.foreach(builder.addNetworkNames)
219230
220231 builder.build
221232 }
222233
223234 def fromProto(proto: Protos.ExtendedContainerInfo.PortMapping): PortMapping =
224235 PortMapping(
225236 proto.getContainerPort,
226237 if (proto.hasHostPort) Some(proto.getHostPort) else None,
227238 proto.getServicePort,
228239 proto.getProtocol,
229240 if (proto.hasName) Some(proto.getName) else None,
230 proto.getLabelsList.map { p => p.getKey -> p.getValue }(collection.breakOut)
241 proto.getLabelsList.map { p => p.getKey -> p.getValue }(collection.breakOut),
242 proto.getNetworkNamesList.toList
231243 )
232244
233245 def toMesos(mapping: Container.PortMapping): Seq[mesos.Protos.ContainerInfo.DockerInfo.PortMapping] = {
234246 def mesosPort(protocol: String, hostPort: Int) = {
235247 mesos.Protos.ContainerInfo.DockerInfo.PortMapping.newBuilder
236248 .setContainerPort (mapping.containerPort)
237249 .setHostPort(hostPort)
238250 .setProtocol(protocol)
239251 .build
240252 }
241253 // we specifically don't want to generate a mesos proto port mapping here if there's no hostport:
242254 //
243255 // 1. hostport is required for the mesos proto, because...
244256 // 2. the mapping is used to set up NAT or port-forwarding from hostport to containerport; noop if hostport is None
245257 mapping.hostPort.fold(Seq.empty[mesos.Protos.ContainerInfo.DockerInfo.PortMapping]) { hp =>
246258 mapping.protocol.split(',').map(mesosPort(_, hp)).toList
247259 }
248260 }
249261
250262 /**
251263 * Build the representation of the PortMapping as a Port proto.
252264 *
253 * @param pm Docker Port Mapping
265 * @param name The Port Mapping name
266 * @param labels The labels to be attached to the Port proto
267 * @param protocol The labels to be attached to the Port proto
254268 * @param effectiveHostPort the effective Mesos Agent port allocated for
255269 * this port mapping.
256 * @return the representation of the PortMapping as a Port proto to be
257 * included in the task's DiscoveryInfo
270 * @return the representation of provided input to be included in the task's DiscoveryInfo
258271 */
259 def toMesosPort(pm: PortMapping, effectiveHostPort: Int): mesos.Protos.Port = {
272 def toMesosPort(name: Option[String], labels: Map[String, String], protocol: String, effectivePort: Int): mesos.Protos.Port = {
260273 val builder = mesos.Protos.Port.newBuilder
261 .setNumber(effectiveHostPort)
262 .setProtocol(pm.protocol)
274 .setNumber(effectivePort)
275 .setProtocol(protocol)
263276
264 pm.name.foreach(builder.setName)
277 name.foreach(builder.setName)
265278
266 if (pm.labels.nonEmpty) {
267 builder.setLabels(pm.labels.toMesosLabels)
279 if (labels.nonEmpty) {
280 builder.setLabels(labels.toMesosLabels)
268281 }
269282
270283 builder.build
271 }
272
273 /**
274 * Generate mesos ports for some endpoint: one port is generated for each endpoint protocol.
275 */
276 def toMesosPorts(ep: Endpoint, effectivePort: Int): Seq[mesos.Protos.Port] = {
277 val builder = mesos.Protos.Port.newBuilder
278 .setNumber(effectivePort)
279 .setName(ep.name)
280
281 if (ep.labels.nonEmpty) {
282 builder.setLabels(ep.labels.toMesosLabels)
283 }
284
285 ep.protocol.map { protocol =>
286 builder.setProtocol(protocol).build
287 }
288284 }
289285
290286}
291287
292288object ParameterSerializer {
293289 def toMesos(param: Parameter): mesos.Protos.Parameter =
294290 mesos.Protos.Parameter.newBuilder
295291 .setKey(param.key)
▲ Show 20 Lines • Show All 102 Lines • Show Last 20 Lines
View Options

src/main/scala/mesosphere/marathon/api/v2/validation/AppValidation.scala

Show All 21 Lines
2222
2323 val portDefinitionsValidator: Validator[Seq[PortDefinition]] = validator[Seq[PortDefinition]] {
2424 portDefinitions =>
2525 portDefinitions is elementsAreUniqueByOptional(_.name, "Port names must be unique.")
2626 portDefinitions is elementsAreUniqueBy(_.port, "Ports must be unique.",
2727 filter = { (port: Int) => port != AppDefinition.RandomPortValue })
2828 }
2929
30 val portMappingsValidator = validator[Seq[ContainerPortMapping]] { portMappings =>
30 val portMappingsIndependentOfNetworks = validator[Seq[ContainerPortMapping]] { portMappings =>
3131 portMappings is elementsAreUniqueByOptional(_.name, "Port names must be unique.")
3232 }
3333
34 def portMappingIsCompatibleWithNetworks(networks: Seq[Network]): Validator[ContainerPortMapping] = {
35 val hostPortNotAllowed = validator[ContainerPortMapping] { mapping =>
36 mapping.hostPort is empty
34 private def portMappingNetworkNameValidator(networkNames: Seq[String]) = validator[ContainerPortMapping] { portMapping =>
35 portMapping.networkNames is every(oneOf(networkNames: _*))
3736 }
38 implied(networks.count(_.mode == NetworkMode.Container) > 1)(hostPortNotAllowed)
37
38 private def portMappingIsCompatibleWithNetworks(networks: Seq[Network]): Validator[ContainerPortMapping] = {
39 val hostPortRequiresNetworkName = isTrue[ContainerPortMapping](
40 AppValidationMessages.NetworkNameRequiredForMultipleContainerNetworks) { mapping =>
jdefUnsubmitted
Done

i wonder how easy-to-read the validation error for this is

41 mapping.hostPort.isEmpty || mapping.networkNames.length == 1
jdefUnsubmitted
Done

networkNamess?

jdefUnsubmitted
Done

shouldn't the second condition be mapping.networkNames.size == 1?

timcharperAuthorUnsubmitted
Done

Yes; I'll add the test.

3942 }
43 implied(networks.count(_.mode == NetworkMode.Container) > 1)(hostPortRequiresNetworkName)
44 }
4045
41 val dockerDockerContainerValidator: Validator[Container] = {
46 def portMappingsValidator(networks: Seq[Network]): Validator[Seq[ContainerPortMapping]] = validator { portMappings =>
47 portMappings is portMappingsIndependentOfNetworks
48 portMappings is every(portMappingIsCompatibleWithNetworks(networks))
49 portMappings is every(portMappingNetworkNameValidator(networks.flatMap(_.name)))
jdefUnsubmitted
Done

portMapping.networkName edge case:

  • list of container networks contains an unnamed network: { mode: container }; normalized to value of --default_network_name
  • portMapping: { hostPort: 0, containerPort: 123 }; should this normalize the missing networkName the same way?

same edge case for endpoints

does it make sense to handle this edge case, or should we keep as-is (only allow mapping.hostPort w/ explicitly named networks when multiple nets are present)? i vote for as-is. either way we should clearly document the systems expected behavior.

timcharperAuthorUnsubmitted
Done

I vote as-is; if you use multiple container networks, you must name them all. Otherwise the flag could change and then validation would fail.

Or... don't specify to use the default container network? This feels like it is getting too complex to validate. Strict seems simpler.

50 }
51
52 def dockerDockerContainerValidator(networks: Seq[Network]): Validator[Container] = {
4253 val validDockerEngineSpec: Validator[DockerContainer] = validator[DockerContainer] { docker =>
4354 docker.image is notEmpty
44 docker.portMappings is valid(optional(portMappingsValidator))
55 docker.portMappings is valid(optional(portMappingsValidator(networks)))
4556 }
4657 validator { (container: Container) =>
4758 container.docker is definedAnd(validDockerEngineSpec)
4859 }
4960 }
5061
5162 val mesosDockerContainerValidator: Validator[Container] = {
5263 val validMesosEngineSpec: Validator[DockerContainer] = validator[DockerContainer] { docker =>
▲ Show 20 Lines • Show All 60 Lines • ▼ Show 20 Line(s)
113124 def validContainer(enabledFeatures: Set[String], networks: Seq[Network]): Validator[Container] = {
114125 def volumesValidator(container: Container): Validator[Seq[AppVolume]] =
115126 isTrue("Volume names must be unique") { (vols: Seq[AppVolume]) =>
116127 val names: Seq[String] = vols.flatMap(_.external.flatMap(_.name))
117128 names.distinct.size == names.size
118129 } and every(valid(validVolume(container, enabledFeatures)))
119130
120131 val validGeneralContainer: Validator[Container] = validator[Container] { container =>
121 container.portMappings is optional(portMappingsValidator and every(portMappingIsCompatibleWithNetworks(networks)))
132 container.portMappings is optional(portMappingsValidator(networks))
122133 container.volumes is volumesValidator(container)
123134 }
124135
125136 val mesosContainerImageValidator = new Validator[Container] {
126137 override def apply(container: Container): Result = {
127138 (container.docker, container.appc, container.`type`) match {
128139 case (Some(_), None, EngineType.Mesos) => validate(container)(mesosDockerContainerValidator)
129140 case (None, Some(_), EngineType.Mesos) => validate(container)(mesosAppcContainerValidator)
130141 case (None, None, EngineType.Mesos) => validate(container)(mesosImagelessContainerValidator)
131142 case _ => Failure(Set(RuleViolation(container, "mesos containers should specify, at most, a single image type", None)))
132143 }
133144 }
134145 }
135146
136147 forAll(
137148 validGeneralContainer,
138 { c: Container => c.`type` == EngineType.Docker } -> dockerDockerContainerValidator,
149 { c: Container => c.`type` == EngineType.Docker } -> dockerDockerContainerValidator(networks),
jdefUnsubmitted
Done

just realized that by adding networkName to ContainerPortMapping that we're actually adding that field to the legacy API as well. we probably want a normalization test to validate that networkName fields specified for the deprecated container.docker.portMappings field is normalized properly to the container.portMappings form (quick scan of the code seems to indicate that this isn't a problem but a unit test would be really great)

139150 { c: Container => c.`type` == EngineType.Mesos } -> mesosContainerImageValidator
140151 )
141152 }
142153
143154 def validVolume(container: Container, enabledFeatures: Set[String]): Validator[AppVolume] = new Validator[AppVolume] {
144155 import state.PathPatterns._
145156 val validHostVolume = validator[AppVolume] { v =>
146157 v.containerPath is valid(notEmpty)
▲ Show 20 Lines • Show All 97 Lines • ▼ Show 20 Line(s)
244255 }
245256
246257 /**
247258 * all validation that touches deprecated app-update API fields goes in here
248259 */
249260 def validateOldAppUpdateAPI: Validator[AppUpdate] = forAll(
250261 validator[AppUpdate] { update =>
251262 update.container is optional(valid(validOldContainerAPI))
252 update.container.flatMap(_.docker.flatMap(_.portMappings)) is optional(portMappingsValidator)
263 update.container.flatMap(_.docker.flatMap(_.portMappings)) is optional(portMappingsIndependentOfNetworks)
253264 update.ipAddress is optional(isTrue(
254265 "ipAddress/discovery is not allowed for Docker containers") { (ipAddress: IpAddress) =>
255266 !(update.container.exists(c => c.`type` == EngineType.Docker) && ipAddress.discovery.nonEmpty)
256267 })
257268 update.uris is optional(every(api.v2.Validation.uriIsValid) and isTrue(
258269 "may not be set in conjunction with fetch") { (uris: Seq[String]) =>
259270 !(uris.nonEmpty && update.fetch.fold(false)(_.nonEmpty))
260271 })
▲ Show 20 Lines • Show All 50 Lines • ▼ Show 20 Line(s)
311322 )
312323
313324 /**
314325 * all validation that touches deprecated app API fields goes in here
315326 */
316327 val validateOldAppAPI: Validator[App] = forAll(
317328 validator[App] { app =>
318329 app.container is optional(valid(validOldContainerAPI))
319 app.container.flatMap(_.docker.flatMap(_.portMappings)) is optional(portMappingsValidator)
330 app.container.flatMap(_.docker.flatMap(_.portMappings)) is optional(portMappingsIndependentOfNetworks)
320331 app.ipAddress is optional(isTrue(
321332 "ipAddress/discovery is not allowed for Docker containers") { (ipAddress: IpAddress) =>
322333 !(app.container.exists(c => c.`type` == EngineType.Docker) && ipAddress.discovery.nonEmpty)
323334 })
324335 app.uris is optional(every(api.v2.Validation.uriIsValid) and isTrue(
325336 "may not be set in conjunction with fetch") { (uris: Seq[String]) => !(uris.nonEmpty && app.fetch.nonEmpty) })
326337 },
327338 isTrue("must not specify both container.docker.network and networks") { app =>
▲ Show 20 Lines • Show All 166 Lines • ▼ Show 20 Line(s)
494505 case _ => false
495506 }
496507 }
497508 (cmd ^ args) || (!(cmd && args) && container)
498509 }
499510}
500511
501512object AppValidation extends AppValidation
513
514object AppValidationMessages {
515 // Note: we should keep this in sync with PodsValidationMessages
jdefUnsubmitted
Done

... and a similar comment here along the lines of the comment you added for PodValidationMessages

516 val NetworkNameRequiredForMultipleContainerNetworks =
jdefUnsubmitted
Done

should this (and the related endpoint comment) read more along these lines?

networkNames must be a single item list when hostPort is specified and more than 1 container network is defined

517 "networkNames must be a single item list when hostPort is specified and more than 1 container network is defined"
518}
View Options

src/main/scala/mesosphere/marathon/api/v2/validation/PodsValidation.scala

Show All 11 Lines
1212
1313// scalastyle:on
1414
1515/**
1616 * Defines implicit validation for pods
1717 */
1818@SuppressWarnings(Array("all")) // wix breaks stuff
1919trait PodsValidation {
20 import PodsValidationMessages._
2021 import EnvVarValidation._
2122 import NameValidation._
2223 import NetworkValidation._
2324 import SchedulingValidation._
2425 import SecretValidation._
2526 import Validation._
2627
2728 val resourceValidator = validator[Resources] { resource =>
▲ Show 20 Lines • Show All 43 Lines • ▼ Show 20 Line(s)
7172 hc.http is optional(httpHealthCheckValidator(endpoints))
7273 hc.tcp is optional(tcpHealthCheckValidator(endpoints))
7374 hc.exec is optional(commandCheckValidator(mesosMasterVersion))
7475 hc is isTrue("Only one of http, tcp, or command may be specified") { hc =>
7576 Seq(hc.http.isDefined, hc.tcp.isDefined, hc.exec.isDefined).count(identity) == 1
7677 }
7778 }
7879
79 def endpointValidator(networks: Seq[Network]) = validator[Endpoint] { endpoint =>
80 // TODO RAML-generated rules should catch these simple things
jdefUnsubmitted
Done

thanks for killing the redundant code

81 endpoint.name.length is between(1, 63)
82 endpoint.name should matchRegexFully(NamePattern)
80 def endpointValidator(networks: Seq[Network]) = {
81 val networkNamess = networks.flatMap(_.name)
82 val hostPortRequiresNetworkName = isTrue[Endpoint](NetworkNameRequiredForMultipleContainerNetworks) { endpoint =>
83 endpoint.hostPort.isEmpty || endpoint.networkNames.length == 1
jdefUnsubmitted
Done

ditto: shouldn't the second condition be endpoint.networkNames.size == 1?

timcharperAuthorUnsubmitted
Done

yep; added another test

83 endpoint.containerPort.getOrElse(1) is between(1, 65535)
84 endpoint.hostPort.getOrElse(0) is between(0, 65535)
84 }
8585
86 val normalValidation = validator[Endpoint] { endpoint =>
87 endpoint.networkNames is every(oneOf(networkNamess: _*))
88
8689 // host-mode networking implies that hostPort is required
8790 endpoint.hostPort is isTrue("is required when using host-mode networking") { hp =>
8891 if (networks.exists(_.mode == NetworkMode.Host)) hp.nonEmpty
8992 else true
9093 }
9194
9295 // host-mode networking implies that containerPort is disallowed
9396 endpoint.containerPort is isTrue("is not allowed when using host-mode networking") { cp =>
Show All 9 Lines
103106
104107 // protocol is an optional field, so we really don't need to validate that is empty/non-empty
105108 // but we should validate that it only contains distinct items
106109 endpoint.protocol is isTrue ("Duplicate protocols within the same endpoint are not allowed") { proto =>
107110 proto == proto.distinct
108111 }
109112 }
110113
114 normalValidation and implied(networks.count(_.mode == NetworkMode.Container) > 1)(hostPortRequiresNetworkName)
115 }
116
111117 val imageValidator = validator[Image] { image =>
112118 image.id.length is between(1, 1024)
113119 }
114120
115121 def volumeMountValidator(volumes: Seq[Volume]): Validator[VolumeMount] = validator[VolumeMount] { volumeMount => // linter:ignore:UnusedParameter
116122 volumeMount.name.length is between(1, 63)
117123 volumeMount.name should matchRegexFully(NamePattern)
118124 volumeMount.mountPath.length is between(1, 1024)
Show All 34 Lines
153159
154160 val scalingValidator: Validator[PodScalingPolicy] = new Validator[PodScalingPolicy] {
155161 override def apply(v1: PodScalingPolicy): Result = v1 match {
156162 case fsf: FixedPodScalingPolicy => fixedPodScalingPolicyValidator(fsf)
157163 case _ => Failure(Set(RuleViolation(v1, "Not a fixed scaling policy", None)))
158164 }
159165 }
160166
161 val endpointNamesUnique: Validator[Pod] = isTrue("Endpoint names are unique") { pod: Pod =>
167 val endpointNamesUnique: Validator[Pod] = isTrue(EndpointNamesMustBeUnique) { pod: Pod =>
162168 val names = pod.containers.flatMap(_.endpoints.map(_.name))
163169 names.distinct.size == names.size
164170 }
165171
166 val endpointContainerPortsUnique: Validator[Pod] = isTrue("Container ports are unique") { pod: Pod =>
172 val endpointContainerPortsUnique: Validator[Pod] = isTrue(ContainerPortsMustBeUnique) { pod: Pod =>
167173 val containerPorts = pod.containers.flatMap(_.endpoints.flatMap(_.containerPort))
168174 containerPorts.distinct.size == containerPorts.size
169175 }
170176
171 val endpointHostPortsUnique: Validator[Pod] = isTrue("Host ports are unique") { pod: Pod =>
177 val endpointHostPortsUnique: Validator[Pod] = isTrue(HostPortsMustBeUnique) { pod: Pod =>
172178 val hostPorts = pod.containers.flatMap(_.endpoints.flatMap(_.hostPort)).filter(_ != 0)
173179 hostPorts.distinct.size == hostPorts.size
174180 }
175181
176182 def podDefValidator(enabledFeatures: Set[String], mesosMasterVersion: SemanticVersion): Validator[Pod] = validator[Pod] { pod =>
177183 PathId(pod.id) as "id" is valid and PathId.absolutePathValidator and PathId.nonEmptyPath
178184 pod.user is optional(notEmpty)
179185 pod.environment is envValidator(strictNameValidation = false, pod.secrets, enabledFeatures)
180 pod.volumes is every(volumeValidator(pod.containers)) and isTrue("volume names are unique") { volumes: Seq[Volume] =>
186 pod.volumes is every(volumeValidator(pod.containers)) and isTrue(VolumeNamesMustBeUnique) { volumes: Seq[Volume] =>
181187 val names = volumes.map(_.name)
182188 names.distinct.size == names.size
183189 }
184190 pod.containers is notEmpty and every(containerValidator(pod, enabledFeatures, mesosMasterVersion))
185 pod.containers is isTrue("container names are unique") { containers: Seq[PodContainer] =>
191 pod.containers is isTrue(ContainerNamesMustBeUnique) { containers: Seq[PodContainer] =>
186192 val names = pod.containers.map(_.name)
187193 names.distinct.size == names.size
188194 }
189195 pod.secrets is empty or (valid(secretValidator) and featureEnabled(enabledFeatures, Features.SECRETS))
190196 pod.networks is valid(ramlNetworksValidator)
191197 pod.scheduling is optional(schedulingValidator)
192198 pod.scaling is optional(scalingValidator)
193199 pod is endpointNamesUnique and endpointContainerPortsUnique and endpointHostPortsUnique
194200 }
195201}
196202
197203object PodsValidation extends PodsValidation {
198204 // TODO: Change this value when mesos supports command checks for pods.
199205 val MinCommandCheckMesosVersion = SemanticVersion(Int.MaxValue, Int.MaxValue, Int.MaxValue)
206}
207
208object PodsValidationMessages {
209 val EndpointNamesMustBeUnique = "endpoint names nust be unique across all containers"
210 val ContainerPortsMustBeUnique = "container ports must be unique across all containers"
211 val HostPortsMustBeUnique = "host ports must be unique across all containers"
212 val VolumeNamesMustBeUnique = "volume names must be unique"
213 val ContainerNamesMustBeUnique = "container names must be unique"
214 // Note: we should keep this in sync with AppValidationMessages
jdefUnsubmitted
Done

maybe a comment to keep this in sync w/ the same error message that we have for apps?

215 val NetworkNameRequiredForMultipleContainerNetworks =
216 "networkNames must be a single item list when hostPort is specified and more than 1 container network is defined"
jdefUnsubmitted
Done

see related portmapping comment

200217}
View Options

src/main/scala/mesosphere/marathon/core/pod/Network.scala

Show All 27 Lines
2828 def hasBridgeNetworking = networks.exists {
2929 case _: BridgeNetwork => true
3030 case _ => false
3131 }
3232 def hasContainerNetworking = networks.exists {
3333 case _: ContainerNetwork => true
3434 case _ => false
3535 }
36
37 /**
38 * Returns whether or not host networking is specified by the network list.
39 *
40 * Note - validations prevent mixed network modes; see ./docs/docs/networking.md
41 */
42 def isHostModeNetworking =
jdefUnsubmitted
Done

yes this!

43 networks.contains(HostNetwork)
44
45 /** Returns all container network names. For bridge or host most networking, returns empty list */
46 def containerNetworkNames: Seq[String] =
47 networks.collect { case ContainerNetwork(name, _) => name }
48
3649 }
3750
3851 val DefaultLabels: Map[String, String] = Map.empty
3952
4053 /** @return deserialized Network, only None if network type is `UNKNOWN` (should never happen in practice) */
4154 def fromProto(net: NetworkDefinition): Option[Network] = {
4255 import NetworkDefinition.Mode._
4356
Show All 26 Lines
7083 case ct: ContainerNetwork =>
7184 builder
7285 .setMode(NetworkDefinition.Mode.CONTAINER)
7386 .setName(ct.name)
7487 .addAllLabels(ct.labels.map{ case (k, v) => Mesos.Label.newBuilder().setKey(k).setValue(v).build() })
7588 }
7689 builder.build()
7790 }
7891}
jdefUnsubmitted
Done

nit: perhaps move into NetworkHelper and rename to names?

timcharperAuthorUnsubmitted
Done

¯\_(ツ)_/¯

I'm not sure what the harm of putting helper methods in the trait's companion.

jdefUnsubmitted
Done

no harm. I just liked the idea of invoking someNetworks.names better than Networks.networkNames(someNetworks). no biggie

View Options

src/main/scala/mesosphere/marathon/core/task/state/NetworkInfo.scala

Show First 20 Lines • Show All 100 Lines • ▼ Show 20 Line(s)
101101 effectiveIpAddress: Option[String],
102102 includeUnresolved: Boolean): Seq[PortAssignment] = {
103103
104104 def fromPortMappings(container: Container): Seq[PortAssignment] = {
105105 import Container.PortMapping
106106 @tailrec
107107 def gen(ports: List[Int], mappings: List[PortMapping], assignments: List[PortAssignment]): List[PortAssignment] = {
108108 (ports, mappings) match {
109 case (hostPort :: xs, PortMapping(containerPort, Some(_), _, _, portName, _) :: rs) =>
109 case (hostPort :: xs, PortMapping(containerPort, Some(_), _, _, portName, _, _) :: rs) =>
110110 // agent port was requested, and we strongly prefer agentIP:hostPort (legacy reasons?)
111111 val assignment = PortAssignment(
112112 portName = portName,
113113 effectiveIpAddress = Option(hostName),
114114 effectivePort = hostPort,
115115 hostPort = Option(hostPort),
116116 containerPort = Option(containerPort)
117117 )
▲ Show 20 Lines • Show All 42 Lines • Show Last 20 Lines
View Options

src/main/scala/mesosphere/marathon/raml/NetworkConversion.scala

Show First 20 Lines • Show All 50 Lines • ▼ Show 20 Line(s)
5151 state.PortDefinition(
5252 port = portDef.port,
5353 protocol = portDef.protocol.value,
5454 name = portDef.name,
5555 labels = portDef.labels
5656 )
5757 }
5858
5959 implicit val portMappingWrites: Writes[state.Container.PortMapping, ContainerPortMapping] = Writes { portMapping =>
jdefUnsubmitted
Done

add support for networkName to the state-to-RAML converter

6060 ContainerPortMapping(
61 networkNames = portMapping.networkNames,
6162 containerPort = portMapping.containerPort,
6263 hostPort = portMapping.hostPort,
6364 labels = portMapping.labels,
6465 name = portMapping.name,
6566 protocol = portMapping.protocol.toRaml[NetworkProtocol],
6667 servicePort = portMapping.servicePort
6768 )
6869 }
6970
7071 implicit val portMappingRamlReader: Reads[ContainerPortMapping, state.Container.PortMapping] = Reads {
71 case ContainerPortMapping(containerPort, hostPort, labels, name, protocol, servicePort) =>
72 case ContainerPortMapping(containerPort, hostPort, labels, name, protocol, servicePort, networkNames) =>
7273 import state.Container.PortMapping._
7374 state.Container.PortMapping(
7475 containerPort = containerPort,
7576 hostPort = hostPort.orElse(defaultInstance.hostPort),
7677 servicePort = servicePort,
7778 protocol = protocol.value,
7879 name = name,
79 labels = labels
80 labels = labels,
81 networkNames = networkNames
8082 )
8183 }
8284
8385 implicit val containerPortMappingProtoRamlWriter: Writes[Protos.ExtendedContainerInfo.PortMapping, ContainerPortMapping] = Writes { mapping =>
jdefUnsubmitted
Done

for completeness, please add support for the networkName field to the proto-to-RAML converter here

8486 ContainerPortMapping(
8587 containerPort = mapping.whenOrElse(_.hasContainerPort, _.getContainerPort, ContainerPortMapping.DefaultContainerPort),
8688 hostPort = mapping.when(_.hasHostPort, _.getHostPort).orElse(ContainerPortMapping.DefaultHostPort),
8789 labels = mapping.whenOrElse(_.getLabelsCount > 0, _.getLabelsList.flatMap(_.fromProto)(collection.breakOut), ContainerPortMapping.DefaultLabels),
8890 name = mapping.when(_.hasName, _.getName).orElse(ContainerPortMapping.DefaultName),
8991 protocol = mapping.when(_.hasProtocol, _.getProtocol).flatMap(NetworkProtocol.fromString).getOrElse(ContainerPortMapping.DefaultProtocol),
90 servicePort = mapping.whenOrElse(_.hasServicePort, _.getServicePort, ContainerPortMapping.DefaultServicePort)
92 servicePort = mapping.whenOrElse(_.hasServicePort, _.getServicePort, ContainerPortMapping.DefaultServicePort),
93 networkNames = mapping.whenOrElse(_.getNetworkNamesList.size > 0, _.getNetworkNamesList.toList, ContainerPortMapping.DefaultNetworkNames)
jdefUnsubmitted
Done

for completeness append? .orElse(ContainerPortMapping.DefaultNetworkName)

jdefUnsubmitted
Done

nit: for consistency, prefer:

networkNames = mapping.whenOrElse(_.getNetworkNamesCount > 0, _.getNetworkNamesList.toList, ContainerPortMapping.DefaultNetworkNames)
timcharperAuthorUnsubmitted
Done

I'll do this because I don't really care; Is there ever a case when we would want an empty list to be anything other than an empty list?

timcharperAuthorUnsubmitted
Done

^ realized the above tone did not come out as intended. To clarify, I'll do this right now, without pushing back, because I'm relatively indifferent. ¯\_(ツ)_/¯ :)

9194 )
9295 }
9396
9497 implicit val dockerPortMappingProtoRamlWriter: Writes[Protos.ExtendedContainerInfo.DockerInfo.ObsoleteDockerPortMapping, ContainerPortMapping] = Writes { mapping =>
9598 ContainerPortMapping(
9699 containerPort = mapping.whenOrElse(_.hasContainerPort, _.getContainerPort, ContainerPortMapping.DefaultContainerPort),
97100 hostPort = mapping.when(_.hasHostPort, _.getHostPort).orElse(ContainerPortMapping.DefaultHostPort),
98101 labels = mapping.whenOrElse(_.getLabelsCount > 0, _.getLabelsList.flatMap(_.fromProto)(collection.breakOut), ContainerPortMapping.DefaultLabels),
Show All 34 Lines
View Options

src/main/scala/mesosphere/marathon/state/AppDefinition.scala

Show First 20 Lines • Show All 77 Lines • ▼ Show 20 Line(s)
7878
7979 override val unreachableStrategy: UnreachableStrategy = AppDefinition.DefaultUnreachableStrategy,
8080
8181 override val killSelection: KillSelection = KillSelection.DefaultKillSelection) extends RunSpec
8282 with plugin.ApplicationSpec with MarathonState[Protos.ServiceDefinition, AppDefinition] {
8383
8484 import mesosphere.mesos.protos.Implicits._
8585
86 /* The following requirements are either validated at the API layer, or precluded by our normalization layer.
87 * However, we had instances of rogue definitions of apps in our tests that were causing related business logic to be
88 * overly complex and handle state that should not exist */
8689 require(networks.nonEmpty, "an application must declare at least one network")
8790
8891 require(
8992 (!networks.exists(!_.eq(HostNetwork))) || portDefinitions.isEmpty,
9093 s"non-host-mode networking ($networks) and ports/portDefinitions ($portDefinitions) are not allowed at the same time")
9194
9295 require(
9396 !(networks.exists(_.eq(HostNetwork)) && container.fold(false)(c => c.portMappings.nonEmpty)),
9497 "port-mappings may not be used in conjunction with host networking")
9598
9699 require(
97100 !(portDefinitions.nonEmpty && container.fold(false)(_.portMappings.nonEmpty)),
98101 "portDefinitions and container.portMappings are not allowed at the same time"
99102 )
100103
104 // Our normalization layer replaces hostPort None to Some(0) for bridge networking
101105 require(
102106 !(networks.hasBridgeNetworking && container.fold(false)(c => c.portMappings.exists(_.hostPort.isEmpty))),
103107 "bridge networking requires that every host-port in a port-mapping is non-empty (but may be zero)")
104108
105109 val portNumbers: Seq[Int] = portDefinitions.map(_.port)
106110
107111 val isResident: Boolean = residency.isDefined
108112
▲ Show 20 Lines • Show All 505 Lines • Show Last 20 Lines
View Options

src/main/scala/mesosphere/marathon/state/Container.scala

Show First 20 Lines • Show All 50 Lines • ▼ Show 20 Line(s)
5151 parameters: Seq[Parameter] = Nil,
5252 forcePullImage: Boolean = false) extends Container {
5353
5454 override def copyWith(portMappings: Seq[PortMapping] = portMappings, volumes: Seq[Volume] = volumes) =
5555 copy(portMappings = portMappings, volumes = volumes)
5656 }
5757
5858 object Docker {
59 val validDockerContainer = validator[Docker] { docker =>
59 implicit val validDockerContainer: Validator[Docker] = validator[Docker] { docker =>
6060 docker.image is notEmpty
61 docker.portMappings is PortMapping.portMappingsValidator
6261 }
6362 }
6463
6564 /**
6665 * @param containerPort The container port to expose
6766 * @param hostPort The host port to bind
6867 * @param servicePort The well-known port for this service
6968 * @param protocol Layer 4 protocol to expose (i.e. "tcp", "udp" or "udp,tcp" for both).
7069 * @param name Name of the service hosted on this port.
7170 * @param labels This can be used to decorate the message with metadata to be
7271 * interpreted by external applications such as firewalls.
72 * @param networkNames Specifies one or more container networks, by name, for which this PortMapping applies.
jdefUnsubmitted
Done

out of date comment

jdefUnsubmitted
Done

please update this comment

jdefUnsubmitted
Done

required is a bit strong. it's an optional field

timcharperAuthorUnsubmitted
Done

isn't it is required in the context specified, though?

jdefUnsubmitted
Done

only requirements apply when hostPort is set (networkNames must be a single-item list) and there are multiple networks in use by the app/pod. there's not enough "context" in this class to add a require or assume assertion because we don't have the list of networks used by the app/pod. but cleaning up the comment a bit would be appreciated

timcharperAuthorUnsubmitted
Not Done

I see what you're saying. Yes, the context is a single PortMapping here, and the requirement concern is wider.

7373 */
7474 case class PortMapping(
7575 containerPort: Int = AppDefinition.RandomPortValue,
7676 hostPort: Option[Int] = None, // defaults to HostPortDefault for BRIDGE mode, None for USER mode
7777 servicePort: Int = AppDefinition.RandomPortValue,
7878 protocol: String = PortMapping.TCP,
7979 name: Option[String] = None,
80 labels: Map[String, String] = Map.empty[String, String])
80 labels: Map[String, String] = Map.empty[String, String],
81 networkNames: Seq[String] = Nil
82 )
8183
8284 object PortMapping {
8385 val TCP = raml.NetworkProtocol.Tcp.value
8486 val UDP = raml.NetworkProtocol.Udp.value
8587 val UDP_TCP = raml.NetworkProtocol.UdpTcp.value
8688 val defaultInstance = PortMapping(name = Option("default"))
8789
8890 val HostPortDefault = AppDefinition.RandomPortValue // HostPortDefault only applies when in BRIDGE mode
89
90 implicit val uniqueProtocols: Validator[Iterable[String]] =
jdefUnsubmitted
Not Done

I think we need the equivalent of this (uniqueProtocols) for the RAML ContainerPortMapping validator in AppValidation

timcharperAuthorUnsubmitted
Not Done

I don't think we do.

In RAML, it's an enum. Only three values are allowed:

docs/docs/rest-api/public/api/v2/types/stringTypes.raml

  39 |   NetworkProtocol:
  40 |     type: string
  41 |     enum: [tcp, udp, "udp,tcp"]
  42 |     description: Protocol of the port (tcp, udp)
  43 |     default: tcp
  44 |   TaskLostBehavior:
jdefUnsubmitted
Not Done

you're right. thanks for keeping me honest :)

91 isTrue[Iterable[String]]("protocols must be unique.") { protocols =>
92 protocols.size == protocols.toSet.size
93 }
94
95 implicit val portMappingValidator = validator[PortMapping] { portMapping =>
96 portMapping.protocol.split(',').toIterable is uniqueProtocols and every(oneOf(TCP, UDP))
97 portMapping.containerPort should be >= 0
98 portMapping.hostPort.each should be >= 0
99 portMapping.servicePort should be >= 0
100 }
101
102 val portMappingsValidator = validator[Seq[PortMapping]] { portMappings =>
103 portMappings is every(valid)
104 portMappings is elementsAreUniqueByOptional(_.name, "Port names must be unique.")
105 }
timcharperAuthorUnsubmitted
Done

Note - the unique name validation happens in AppValidation. The rest of these are taken care of by RAML itself.

10691 }
10792
10893 case class Credential(
10994 principal: String,
11095 secret: Option[String] = None)
11196
11297 case class MesosDocker(
11398 volumes: Seq[Volume] = Seq.empty,
▲ Show 20 Lines • Show All 64 Lines • Show Last 20 Lines
View Options

src/main/scala/mesosphere/mesos/NetworkScope.scala

  • This file was deleted.
This file was completely deleted. Show File Contents
View Options

src/main/scala/mesosphere/mesos/PortDiscovery.scala

11package mesosphere.mesos
22
3import mesosphere.marathon.api.serialization.{ PortDefinitionSerializer, PortMappingSerializer }
4import mesosphere.marathon.core.pod.HostNetwork
3import mesosphere.marathon.api.serialization.PortMappingSerializer
54import mesosphere.marathon.raml.Endpoint
6import mesosphere.marathon.state.AppDefinition
5import mesosphere.marathon.state.{ AppDefinition, PortDefinition }
6import mesosphere.marathon.state.Container.PortMapping
77import org.apache.mesos.Protos.Port
8import mesosphere.marathon.core.pod.Network
89
910import scala.collection.immutable.Seq
1011
11trait PortDiscovery {
12object PortDiscovery {
13 /**
14 * Generic interface to read either an EndPoint or a PortMapping. This allows us to write generically against these
15 * two separate, but similar, data structures.
16 *
17 * TODO - extract and use more widely to fold away duplicate code paths dealing with these two separate but similar
jdefUnsubmitted
Not Done

this will be difficult without taking into account servicePort and requirePorts. let's chat about how to move forward

18 * data structures
19 */
20 private trait PortMappingReader[T] {
21 def name(t: T): Option[String]
22 def labels(t: T): Map[String, String]
23 def protocols(t: T): Seq[String]
24 def networkNames(t: T): Seq[String]
25 def containerPort(t: T): Option[Int]
26 def hostPort(t: T): Option[Int]
27 }
28 private object PortMappingReader {
jdefUnsubmitted
Done

where's the implicit reader object for PortDefinition?

timcharperAuthorUnsubmitted
Done

added

29 implicit object EndpointReader extends PortMappingReader[Endpoint] {
30 def name(ep: Endpoint) = Some(ep.name)
31 def labels(ep: Endpoint) = ep.labels
32 def protocols(ep: Endpoint) = ep.protocol
33 def networkNames(ep: Endpoint) = ep.networkNames
34 def containerPort(ep: Endpoint) = ep.containerPort
35 def hostPort(ep: Endpoint) = ep.hostPort
36 }
1237
38 implicit object ContainerPortMappingReader extends PortMappingReader[PortMapping] {
39 def name(portMapping: PortMapping) = portMapping.name
40 def labels(portMapping: PortMapping) = portMapping.labels
41 def protocols(portMapping: PortMapping) = portMapping.protocol.split(',').to[Seq]
42 def networkNames(portMapping: PortMapping) = portMapping.networkNames
43 def containerPort(portMapping: PortMapping) = Some(portMapping.containerPort)
44 def hostPort(portMapping: PortMapping) = portMapping.hostPort
45 }
46
47 implicit object PortDefinitionReader extends PortMappingReader[PortDefinition] {
48 def name(portDefinition: PortDefinition) = portDefinition.name
49 def labels(portDefinition: PortDefinition) = portDefinition.labels
50 def protocols(portDefinition: PortDefinition) = portDefinition.protocol.split(',').to[Seq]
51 def networkNames(portDefinition: PortDefinition) = Nil
52 def containerPort(portDefinition: PortDefinition) = None
53 def hostPort(portDefinition: PortDefinition) = Some(portDefinition.port)
54 }
55 }
56
1357 /**
14 * @param hostModeNetworking is true if we're only using host networking (vs. bridged or container networking)
15 * @param endpoints are assumed to have had wildcard ports (e.g. 0) filled in with actual port numbers
58 * Generate mesos Port discovery proto for each PortMapping / Endpoint entry in order to facilitate port discovery
59 * via mesos.
60 *
61 * If hostPort is nonEmpty, generate a single record with the network-scope of "host"
62 *
63 * Otherwise, generate a record for the cartesian expansion of every protocol AND associated container network. For
64 * example, if an entry specifies two protocols and 3 container networks, then we generate 6 mesos Port proto
65 * records. Records have a network-scope of "container", and, additionally, specify the associated container network
66 * via the network-name label.
67 *
68 * @param networks The networks associated with the appSpec / pod
69 * @param endpoints Seq of Endpoint or container PortMapping
70 *
71 * @return Seq of mesos Port proto records, intended to be used to support port discovery via mesos
1672 */
17 def generate(hostModeNetworking: Boolean, endpoints: Seq[Endpoint]): Seq[Port] =
18 if (!hostModeNetworking) {
73 def generate[T](networks: Seq[Network], mappings: Seq[T])(implicit r: PortMappingReader[T]): Seq[Port] = {
74 if (!networks.isHostModeNetworking) {
jdefUnsubmitted
Done

unclear why this method signature had to change..? (a bool is completely sufficient, yes?)

timcharperAuthorUnsubmitted
Done

bool is sufficient; we had implemented how to judge if networks was host networking twice (see line 42 on the left side)

timcharperAuthorUnsubmitted
Done

But... there's more than one way to unify... I'm generally a fan of passing the least amount of data to do the job. There was a period when I was going to infer networkName if it was not specified (for discovery reasons) but ultimately decided that might be a bad idea. I don't know. This feature feels under-designed.

1975 // The run spec uses bridge and user modes with portMappings, use them to create the Port messages.
2076 // Just like apps, we prefer to generate network-scope=host when there's a hostPort available.
21 endpoints.flatMap {
77 mappings.flatMap { mp =>
jdefUnsubmitted
Done

nit: networkName is unused for building the discovery labels when network-scope=host

22 case (ep @ Endpoint(_, Some(_), Some(hostPort), _, _)) =>
23 val updatedEp = ep.copy(labels = ep.labels + NetworkScope.Host.discovery)
24 PortMappingSerializer.toMesosPorts(updatedEp, hostPort)
25 case (ep @ Endpoint(_, Some(containerPort), None, _, _)) =>
26 val updatedEp = ep.copy(labels = ep.labels + NetworkScope.Container.discovery)
27 PortMappingSerializer.toMesosPorts(updatedEp, containerPort)
28 case ep =>
29 throw new IllegalStateException(s"unexpected combination of network mode and endpoint ports for endpoint $ep")
78 (r.containerPort(mp), r.hostPort(mp)) match {
79 case (Some(_), Some(hostPort)) =>
80 r.protocols(mp).map { protocol =>
81 PortMappingSerializer.toMesosPort(
82 name = r.name(mp),
83 labels = r.labels(mp) ++ discoveryLabelsHost,
84 protocol = protocol,
85 effectivePort = hostPort)
86 }
87 case (Some(containerPort), None) =>
88 for {
89 protocol <- r.protocols(mp)
jdefUnsubmitted
Not Done

I think we're getting away from using for constructs in favor of map/flatMap

timcharperAuthorUnsubmitted
Not Done

we should update our contributing.md file, in which it explicitly says:

BAD:
xs.flatMap(_.flatMap(_.map(_.z))) // replace with a for comprehension
jdefUnsubmitted
Done

agreed. JIRA?

timcharperAuthorUnsubmitted
Not Done

JIRA MARATHON-7244 Update contributing.md to reflect our consensus on for comprehensions

timcharperAuthorUnsubmitted
Not Done

great; earlier, we discussed only rejecting style changes that were documented. Does that apply in this case?

jdefUnsubmitted
Not Done

i won't reject for this

90 networkName <- if (r.networkNames(mp).isEmpty) networks.containerNetworkNames else r.networkNames(mp)
91 } yield {
92 PortMappingSerializer.toMesosPort(
93 name = r.name(mp),
94 labels = r.labels(mp) ++ discoveryLabelsContainer(networkName),
95 protocol = protocol,
96 effectivePort = containerPort)
97 }
98 case _ =>
99 throw new IllegalStateException(
100 s"unexpected combination of network mode and endpoint ports for ${mp.getClass.getSimpleName} $mp")
101 }
30102 }(collection.breakOut)
31103 } else {
32 // The port numbers are the allocated ports, we need to overwrite them the port numbers assigned to this particular task.
33104 // network-scope is assumed to be host, no need for an additional scope label here.
34 endpoints.flatMap { ep =>
35 val hostPort: Int = ep.hostPort.getOrElse(throw new IllegalStateException(
36 "expected non-empty host port in conjunction with host networking"
37 ))
38 PortMappingSerializer.toMesosPorts(ep, hostPort)
39 }(collection.breakOut)
40 }
105 for {
106 ep <- mappings
41107
42 def generate(runSpec: AppDefinition, hostPorts: Seq[Option[Int]]): Seq[Port] = {
43 if (runSpec.networks.exists(_ != HostNetwork)) {
44 runSpec.container.withFilter(_.portMappings.nonEmpty).map { c =>
45 // The run spec uses bridge and user modes with portMappings, use them to create the Port messages
46 c.portMappings.zip(hostPorts).collect {
47 case (portMapping, None) =>
48 // No host port has been defined. See PortsMatcher.mappedPortRanges, use container port instead.
49 val updatedPortMapping =
50 portMapping.copy(labels = portMapping.labels + NetworkScope.Container.discovery)
51 PortMappingSerializer.toMesosPort(updatedPortMapping, portMapping.containerPort)
52 case (portMapping, Some(hostPort)) =>
53 // When there's a host port, advertise that for discovery because it may result in better
54 // network performance (using host ports may be faster than bridged/NATd ports).
55 val updatedPortMapping = portMapping.copy(labels = portMapping.labels + NetworkScope.Host.discovery)
56 PortMappingSerializer.toMesosPort(updatedPortMapping, hostPort)
108 hostPort: Int = r.hostPort(ep).getOrElse(throw new IllegalStateException(
109 "expected non-empty host port in conjunction with host networking"))
110
111 protocol <- r.protocols(ep)
112 } yield PortMappingSerializer.toMesosPort(
113 name = r.name(ep),
114 labels = r.labels(ep),
115 protocol = protocol,
116 effectivePort = hostPort)
57117 }
58 }.getOrElse(Nil) // no port mappings **and** non-host networking? then you don't have ports...
118 }
119
120 /**
jdefUnsubmitted
Done

if you added a condition to the case statement: if portMapping.hostPort.isEmpty => ...

jdefUnsubmitted
Done

.. and here if portMapping.hostPort.nonEmpty => ...

jdefUnsubmitted
Done

and then a catch-all:

case unsupported => throw new IllegalStateException(s"unsupported combination of portMapping $portMapping and host port allocation")

... this would probably flag invalid unit test fixtures.

jdefUnsubmitted
Done

@param runSpec adds no value

timcharperAuthorUnsubmitted
Done

lol

jdefUnsubmitted
Done

this func lost support for portDefinitions? that's not right...

121 * See [[generate]]
122 *
123 * @param networks the networks specified for the pod
124 * @param endpoints are assumed to have had wildcard ports (e.g. 0) filled in with actual port numbers
125 */
126 def generateForPod(networks: Seq[Network], endpoints: Seq[Endpoint]): Seq[Port] =
127 generate(networks, endpoints)
128
129 /**
130 * See [[generate]]
131 *
132 * @param hostPorts the hostPorts after resource matching
133 */
134 def generateForApp(runSpec: AppDefinition, hostPorts: Seq[Option[Int]]): Seq[Port] =
135 if (!runSpec.networks.isHostModeNetworking) {
136 generate(
137 runSpec.networks,
138 runSpec.container
139 .fold(Seq.empty[(PortMapping, Option[Int])]) { c => c.portMappings.zip(hostPorts) }
140 .map {
141 case (portMapping, assignment) =>
142 if (portMapping.hostPort.isEmpty != assignment.isEmpty)
143 throw new IllegalStateException(
144 s"unsupported combination of portMapping ${portMapping} and host port allocation")
145 else
146 portMapping.copy(hostPort = assignment)
147 })
59148 } else {
60 // Serialize runSpec.portDefinitions to protos. The port numbers are the service ports, we need to
61 // overwrite them the port numbers assigned to this particular task.
62 // network-scope is assumed to be host, no need for an additional scope label here.
63 runSpec.portDefinitions.zip(hostPorts).collect {
64 case (portDefinition, Some(hostPort)) =>
65 PortDefinitionSerializer.toMesosProto(portDefinition).map(_.toBuilder.setNumber(hostPort).build)
66 }.flatten
149 // The port numbers are the allocated ports, we need to overwrite them the port numbers assigned to this particular task.
150 generate(
151 runSpec.networks,
152 runSpec.portDefinitions.zip(hostPorts).map {
153 case (portDefinition, Some(assignment)) =>
154 portDefinition.copy(port = assignment)
155 case _ =>
156 throw new IllegalStateException(
157 "illegal portDefinition without host assignment")
jdefUnsubmitted
Done

nit: s not needed, no substitution

158 })
159
67160 }
68 }
69}
161 val NetworkScopeHost = "host"
162 val NetworkScopeContainer = "container"
70163
71object PortDiscovery extends PortDiscovery
164 val NetworkScopeLabel = "network-scope"
165 val NetworkNameLabel = "network-name"
166
167 /**
168 * @return Seq of appropriate network discovery labels for network-scope (all networks) and network-name (container
169 * networks)
170 */
171 private val discoveryLabelsHost: Seq[(String, String)] =
jdefUnsubmitted
Done

mind filing a follow-up JIRA to update our service discovery docs to mention these things? feels like tech debt since we never documented the original network-scope and I think we've seen enough scope creep for this ticket already

timcharperAuthorUnsubmitted
Done

JIRA MARATHON-7241 Document network-scope

172 Seq(NetworkScopeLabel -> NetworkScopeHost)
173
174 private def discoveryLabelsContainer(networkName: String): Seq[(String, String)] =
175 Seq(
176 NetworkScopeLabel -> NetworkScopeContainer,
177 NetworkNameLabel -> networkName)
178}
View Options

src/main/scala/mesosphere/mesos/TaskBuilder.scala

Show First 20 Lines • Show All 112 Lines • ▼ Show 20 Line(s)
113113 runSpec: AppDefinition,
114114 hostPorts: Seq[Option[Int]]): org.apache.mesos.Protos.DiscoveryInfo = {
115115
116116 val discoveryInfoBuilder = org.apache.mesos.Protos.DiscoveryInfo.newBuilder
117117 discoveryInfoBuilder.setName(runSpec.id.toHostname)
118118 discoveryInfoBuilder.setVisibility(org.apache.mesos.Protos.DiscoveryInfo.Visibility.FRAMEWORK)
119119
120120 val portsProto = org.apache.mesos.Protos.Ports.newBuilder
121 portsProto.addAllPorts(PortDiscovery.generate(runSpec, hostPorts))
121 portsProto.addAllPorts(PortDiscovery.generateForApp(runSpec, hostPorts))
122122
123123 discoveryInfoBuilder.setPorts(portsProto)
124124 discoveryInfoBuilder.build
125125 }
126126
127127 protected def computeContainerInfo(hostPorts: Seq[Option[Int]], taskId: Task.Id): Option[ContainerInfo] = {
128128 if (runSpec.container.isEmpty && !runSpec.networks.hasNonHostNetworking) {
129129 None
▲ Show 20 Lines • Show All 138 Lines • Show Last 20 Lines
View Options

src/main/scala/mesosphere/mesos/TaskGroupBuilder.scala

Show All 35 Lines
3636 config: BuilderConfig,
3737 runSpecTaskProcessor: RunSpecTaskProcessor,
3838 resourceMatch: ResourceMatcher.ResourceMatch
3939 ): (mesos.ExecutorInfo, mesos.TaskGroupInfo, Seq[Option[Int]], Instance.Id) = {
4040 val instanceId = newInstanceId(podDefinition.id)
4141
4242 val allEndpoints = podDefinition.containers.flatMap(_.endpoints)
4343
44 assume(
45 allEndpoints.size == resourceMatch.hostPorts.size,
46 "expected total number of endpoints to match number of optional host ports")
44 val mesosNetworks = buildMesosNetworks(podDefinition.networks, allEndpoints, resourceMatch.hostPorts)
4745
48 val portMappings = computePortMappings(allEndpoints, resourceMatch.hostPorts)
49
5046 val executorInfo = computeExecutorInfo(
5147 podDefinition,
5248 resourceMatch.portsMatch,
53 portMappings,
49 mesosNetworks,
5450 instanceId,
5551 offer.getFrameworkId)
5652
5753 val endpointAllocationsPerContainer: Map[String, Seq[Endpoint]] =
5854 podDefinition.containers.flatMap { c =>
5955 c.endpoints.map(c.name -> _)
6056 }.zip(resourceMatch.hostPorts).groupBy { case (t, _) => t._1 }.map {
6157 case (k, s) =>
Show All 15 Lines
7773 runSpecTaskProcessor.taskGroup(podDefinition, executorInfo, taskGroup)
7874
7975 (executorInfo.build, taskGroup.build, resourceMatch.hostPorts, instanceId)
8076 }
8177
8278 // The resource match provides us with a list of host ports.
8379 // Each port mapping corresponds to an item in that list.
8480 // We use that list to swap the dynamic ports (ports set to 0) with the matched ones.
85 private[this] def computePortMappings(
81 private[mesos] def buildMesosNetworks(
82 networks: Seq[Network],
8683 endpoints: Seq[raml.Endpoint],
87 hostPorts: Seq[Option[Int]]): Seq[mesos.NetworkInfo.PortMapping] = {
84 hostPorts: Seq[Option[Int]]): Seq[mesos.NetworkInfo] = {
8885
89 endpoints.zip(hostPorts).collect {
86 assume(
87 endpoints.size == hostPorts.size,
88 "expected total number of endpoints to match number of optional host ports")
89 val portMappings = endpoints.iterator
90 .zip(hostPorts.iterator)
91 .collect {
9092 case (endpoint, Some(hostPort)) =>
9193 val portMapping = mesos.NetworkInfo.PortMapping.newBuilder
9294 .setHostPort(hostPort)
9395
9496 if (endpoint.containerPort.forall(_ == 0)) {
9597 portMapping.setContainerPort(hostPort)
9698 } else {
9799 endpoint.containerPort.foreach(portMapping.setContainerPort)
98100 }
99101
100102 // While the protocols in RAML may be declared in a list, Mesos expects a
101103 // port mapping for every single protocol. If protocols are set, a port mapping
102104 // will be created for every protocol in the list.
103105 if (endpoint.protocol.isEmpty) {
104 Seq(portMapping.build)
106 Seq((endpoint.networkNames, portMapping.build))
105107 } else {
106108 endpoint.protocol.map { protocol =>
107 portMapping.setProtocol(protocol).build
109 (endpoint.networkNames, portMapping.setProtocol(protocol).build)
108110 }
109111 }
110 }.flatten
111112 }
113 .flatten
114 .toList
112115
116 networks.collect{
117 case containerNetwork: ContainerNetwork =>
jdefUnsubmitted
Done

looks like D201 left out support for bridge networks and port mappings. please add it here

timcharperAuthorUnsubmitted
Done

filed ticket JIRA MARATHON-7204 Bridge Network portMapping support not implemented for Pods

118 val b = mesos.NetworkInfo.newBuilder
119 .setName(containerNetwork.name)
120 .setLabels(containerNetwork.labels.toMesosLabels)
121 portMappings.foreach {
122 case (networkNames, pm) =>
123 // if networkNames is empty, then it means associate with all container networks
jdefUnsubmitted
Done

I really had to think about this line. Probably because forall still throws me off when I see it. The meaning here is "if networkNames is empty or else networkNames is a single item list containing containerNetwork.name" -- which is correct. Just took me a min to reason about. Maybe a comment would be nice? (or not if you think that's superfluous)

124 if (networkNames.forall(_ == containerNetwork.name))
125 b.addPortMappings(pm)
126 }
127 b.build
128 }
129 }
130
113131 private[this] def computeTaskInfo(
114132 container: MesosContainer,
115133 podDefinition: PodDefinition,
116134 offer: mesos.Offer,
117135 instanceId: Instance.Id,
118136 hostPorts: Seq[Option[Int]],
119137 config: BuilderConfig,
120138 portAssignments: Seq[PortAssignment]): mesos.TaskInfo.Builder = {
Show All 36 Lines
157175 }
158176
159177 builder
160178 }
161179
162180 private[this] def computeExecutorInfo(
163181 podDefinition: PodDefinition,
164182 portsMatch: PortsMatch,
165 portMappings: Seq[mesos.NetworkInfo.PortMapping],
183 mesosNetworks: Seq[mesos.NetworkInfo],
166184 instanceId: Instance.Id,
167185 frameworkId: mesos.FrameworkID): mesos.ExecutorInfo.Builder = {
168186 val executorID = mesos.ExecutorID.newBuilder.setValue(instanceId.executorIdString)
169187
170188 val executorInfo = mesos.ExecutorInfo.newBuilder
171189 .setType(mesos.ExecutorInfo.Type.DEFAULT)
172190 .setExecutorId(executorID)
173191 .setFrameworkId(frameworkId)
174192
175193 executorInfo.addResources(scalarResource("cpus", podDefinition.executorResources.cpus))
176194 executorInfo.addResources(scalarResource("mem", podDefinition.executorResources.mem))
177195 executorInfo.addResources(scalarResource("disk", podDefinition.executorResources.disk))
178196 executorInfo.addResources(scalarResource("gpus", podDefinition.executorResources.gpus.toDouble))
179197 executorInfo.addAllResources(portsMatch.resources)
180198
181199 if (podDefinition.networks.nonEmpty || podDefinition.volumes.nonEmpty) {
182200 val containerInfo = mesos.ContainerInfo.newBuilder
183201 .setType(mesos.ContainerInfo.Type.MESOS)
184202
185 podDefinition.networks.collect{
186 case containerNetwork: ContainerNetwork =>
187 mesos.NetworkInfo.newBuilder
188 .setName(containerNetwork.name)
189 .setLabels(containerNetwork.labels.toMesosLabels)
190 .addAllPortMappings(portMappings)
191 }.foreach{ networkInfo =>
192 containerInfo.addNetworkInfos(networkInfo)
193 }
203 mesosNetworks.foreach(containerInfo.addNetworkInfos(_))
194204
195205 podDefinition.volumes.collect{
196206 // see the related code in computeContainerInfo
197207 case e: EphemeralVolume =>
198208 mesos.Volume.newBuilder()
199209 .setMode(mesos.Volume.Mode.RW) // if not RW, then how do containers plan to share anything?
200210 .setSource(mesos.Volume.Source.newBuilder()
201211 .setType(mesos.Volume.Source.Type.SANDBOX_PATH)
▲ Show 20 Lines • Show All 237 Lines • ▼ Show 20 Line(s)
439449 private[this] def scalarResource(name: String, value: Double): mesos.Resource.Builder = {
440450 mesos.Resource.newBuilder
441451 .setName(name)
442452 .setType(mesos.Value.Type.SCALAR)
443453 .setScalar(mesos.Value.Scalar.newBuilder.setValue(value))
444454 }
445455
446456 private def taskDiscovery(pod: PodDefinition, endpoints: Seq[Endpoint]): mesos.DiscoveryInfo = {
447 val ports = PortDiscovery.generate(pod.networks.contains(HostNetwork), endpoints)
457 val ports = PortDiscovery.generateForPod(pod.networks, endpoints)
448458 mesos.DiscoveryInfo.newBuilder.setPorts(mesos.Ports.newBuilder.addAllPorts(ports))
449459 .setName(pod.id.toHostname)
450460 .setVisibility(org.apache.mesos.Protos.DiscoveryInfo.Visibility.FRAMEWORK)
451461 .build
452462 }
453463}
View Options

src/test/scala/mesosphere/UnitTest.scala

11package mesosphere
22
33import java.util.concurrent.{ LinkedBlockingDeque, TimeUnit }
44
55import akka.actor.{ ActorSystem, Scheduler }
66import akka.stream.{ ActorMaterializer, Materializer }
77import akka.testkit.{ TestActor, TestActorRef, TestKitBase }
88import akka.util.Timeout
99import com.typesafe.config.{ Config, ConfigFactory }
1010import com.typesafe.scalalogging.StrictLogging
11import com.wix.accord.{ Failure, Result }
11import com.wix.accord.{ Failure, Result, Success }
1212import kamon.Kamon
1313import mesosphere.marathon.Normalization
1414import mesosphere.marathon.ValidationFailedException
1515import mesosphere.marathon.api.v2.Validation
1616import mesosphere.marathon.test.{ ExitDisabledTest, Mockito }
17import org.scalatest.matchers.{ Matcher, MatchResult }
1718import org.scalatest._
1819import org.scalatest.concurrent.{ JavaFutures, ScalaFutures }
1920import org.scalatest.time.{ Seconds, Span }
21import mesosphere.marathon.api.v2.ValidationHelper
2022
2123import scala.concurrent.ExecutionContextExecutor
2224
2325/**
2426 * Tests which are still unreliable should be marked with this tag until
2527 * they sufficiently pass on master. Prefer this over ignored.
2628 */
2729object Unstable extends Tag("mesosphere.marathon.UnstableTest")
Show All 39 Lines
6769 case x => x
6870 }
6971
7072 def withValidationClue[T](f: => T): T = scala.util.Try { f }.recover {
7173 // handle RAML validation errors
7274 case vfe: ValidationFailedException => fail(vfe.failure.violations.toString())
7375 case th => throw th
7476 }.get
77
78 def containViolation(tuple: (String, String)): Matcher[Result] = containViolation(tuple._1, tuple._2)
79
80 def containViolation(path: String, message: String): Matcher[Result] = {
jdefUnsubmitted
Done

nit: can we have a better name for template .. like message?

81 Matcher {
82 case Success =>
83 MatchResult(
84 false,
85 s"result had no violations; expected ${path} -> ${message}",
86 s"result was success")
87
88 case f: Failure =>
89 val violations = ValidationHelper.getAllRuleConstrains(f)
90
91 MatchResult(
92 violations.exists { v =>
93 v.path.contains(path) && v.message == message
94 },
95 s"Violations:\n${violations.mkString("\n")} did not contain ${path} -> ${message}",
96 s"Violation contains ${path} -> ${message}"
97 )
98 }
99 }
75100}
76101
77102/**
78103 * Base trait for all unit tests in WordSpec style with common matching/before/after and Option/Try/Future
79104 * helpers all mixed in.
80105 */
81106trait UnitTestLike extends WordSpecLike
82107 with GivenWhenThen
▲ Show 20 Lines • Show All 63 Lines • Show Last 20 Lines
View Options

src/test/scala/mesosphere/marathon/api/serialization/ContainerSerializerTest.scala

  • This file was added.
1package mesosphere.marathon
2package api.serialization
3
4import mesosphere.UnitTest
5import mesosphere.marathon.state.Container.PortMapping
6import mesosphere.marathon.state.Container
7import scala.collection.JavaConverters._
8import mesosphere.marathon.core.pod.ContainerNetwork
9import org.scalatest.Inside
10
11class ContainerSerializerTest extends UnitTest with Inside {
12 "network toMesos serializer" when {
13 "a single container network is defined" should {
14 "assign the portMappings to the only defined network" in {
jdefUnsubmitted
Done

would like to see a test for bridge network too; could reduce redundancy between this and a bridge test via behave like

timcharperAuthorUnsubmitted
Done

this seems out of scope for the change at hand; should we file an issue and do in a separate patch?

timcharperAuthorUnsubmitted
Done

JIRA MARATHON-7204 Bridge Network portMapping support not implemented for Pods

15 val networks = List(ContainerNetwork("network"))
16 val container = Container.Mesos(Nil, List(PortMapping(hostPort = Some(1000))))
17
18 val result = ContainerSerializer.toMesos(networks, container, "mesos-bridge")
19
20 val Seq(networkInfo) = result.getNetworkInfosList.asScala.toList
21 networkInfo.getName shouldBe ("network")
22 inside(networkInfo.getPortMappingsList.asScala.toList) {
23 case Seq(portMapping) =>
24 portMapping.getHostPort shouldBe 1000
25 }
26 }
27 }
28
29 "multiple container networks are defined" should {
30 "assign the portMappings to the specified network" in {
31 val networks = List(ContainerNetwork("network-1"), ContainerNetwork("network-2"))
32 val container = Container.Mesos(Nil, List(
33 PortMapping(hostPort = Some(1000), networkNames = List("network-1")),
34 PortMapping(hostPort = Some(1001), networkNames = List("network-2"))))
35
36 val result = ContainerSerializer.toMesos(networks, container, "mesos-bridge")
37
38 val Seq(networkInfo1, networkInfo2) = result.getNetworkInfosList.asScala.toList
39 networkInfo1.getName shouldBe ("network-1")
40 inside(networkInfo1.getPortMappingsList.asScala.toList) {
41 case Seq(portMapping) =>
42 portMapping.getHostPort shouldBe 1000
43 }
44
45 networkInfo2.getName shouldBe ("network-2")
46 inside(networkInfo2.getPortMappingsList.asScala.toList) {
47 case Seq(portMapping) =>
48 portMapping.getHostPort shouldBe 1001
49 }
50 }
51 }
52 }
53}
View Options

src/test/scala/mesosphere/marathon/api/v2/AppNormalizationTest.scala

Show First 20 Lines • Show All 190 Lines • ▼ Show 20 Line(s)
191191 portMappings = Some(Nil)
192192 )),
193193 networks = Seq(Network(mode = NetworkMode.ContainerBridge)),
194194 unreachableStrategy = Some(UnreachableEnabled.Default)
195195 )
196196 legacyDockerApp.normalize should be(normalDockerApp)
197197 }
198198
199 "preserves networkNames field" in {
200 val legacyDockerApp = App(
201 "/foo",
202 container = Some(Container(
203 `type` = EngineType.Docker,
204 docker = Some(DockerContainer(
205 image = "image0",
206 portMappings = Some(Seq(
207 ContainerPortMapping(
208 containerPort = 80,
209 networkNames = List("1"))))))
210 )),
211 networks = Seq(Network(mode = NetworkMode.Container, name = Some("1")))
212 )
213
214 val Some(Seq(portMapping)) = legacyDockerApp.normalize.container.flatMap(_.portMappings)
jdefUnsubmitted
Done

thanks!

215 portMapping shouldBe ContainerPortMapping(
216 containerPort = 80,
217 networkNames = List("1"))
218 }
219
199220 "legacy docker app specifies ipAddress and HOST networking" in {
200221 val legacyDockerApp = App(
201222 "/foo",
202223 container = Some(Container(
203224 `type` = EngineType.Docker,
204225 docker = Some(DockerContainer(network = Some(DockerNetwork.Host), image = "image0", portMappings = Some(Nil)))
205226 )),
206227 ipAddress = Some(IpAddress(networkName = Some("mesos-bridge")))
▲ Show 20 Lines • Show All 403 Lines • Show Last 20 Lines
View Options

src/test/scala/mesosphere/marathon/api/v2/json/AppDefinitionSchemaJSONTest.scala

Show First 20 Lines • Show All 86 Lines • ▼ Show 20 Line(s)
8787 | "ports": [
8888 | {"name": "dns", "number": 53, "protocol": "foo"}
8989 | ]
9090 | }
9191 | }
9292 |}
9393 """.stripMargin
9494
95 Then("validation should succeed")
95 Then("validation should fail")
9696 MarathonTestHelper.validateJsonSchemaForString(json, valid = false)
9797 }
9898
9999 "command constrains WITH array of arrays of strings should succeed" in {
100100 Given("constrains WITH nested arrays of strings")
101101 val json =
102102 """
103103 |{
▲ Show 20 Lines • Show All 46 Lines • ▼ Show 20 Line(s)
150150 "command constrains WITH array of one string should fail" in {
151151 Given("constrains WITH nested array of something")
152152 val json =
153153 """
154154 |{
155155 | "id": "/test",
156156 | "cmd": "echo hi",
157157 | "constraints": [["hostname"]]
158 |}
159 """.stripMargin
160
161 Then("validation should fail")
162 MarathonTestHelper.validateJsonSchemaForString(json, valid = false)
163 }
164
165 "portMapping with valid networkNames should succeed" in {
166 Given("portMapping with invalid networkNames")
167 val json =
168 """
169 |{
170 | "id": "/test",
171 | "cmd": "echo hi",
172 | "container": {
173 | "type": "MESOS",
174 | "portMappings": [
175 | {"hostPort": 0, "networkNames": ["valid-name"]}
jdefUnsubmitted
Not Done

this won't pass once the schema is corrected

176 | ]
177 | }
178 |}
179 """.stripMargin
180
181 Then("validation should succeed")
182 MarathonTestHelper.validateJsonSchemaForString(json, valid = true)
183 }
184
185 "portMapping with invalid network name should fail" in {
186 Given("portMapping with invalid networkNames")
187 val json =
188 """
189 |{
190 | "id": "/test",
191 | "cmd": "echo hi",
192 | "container": {
193 | "type": "MESOS",
194 | "portMappings": [
195 | {"hostPort": 0, "networkNames": ["invalid name"]}
jdefUnsubmitted
Done

ditto: should be ["invalid name"]

196 | ]
197 | }
198 |}
199 """.stripMargin
200
201 Then("validation should fail")
202 MarathonTestHelper.validateJsonSchemaForString(json, valid = false)
203 }
204
205 "portMapping with non-array should fail" in {
206 Given("portMapping with invalid networkNames")
207 val json =
208 """
209 |{
210 | "id": "/test",
211 | "cmd": "echo hi",
212 | "container": {
213 | "type": "MESOS",
214 | "portMappings": [
215 | {"hostPort": 0, "networkNames": "invalid name"}
216 | ]
217 | }
158218 |}
159219 """.stripMargin
160220
161221 Then("validation should fail")
162222 MarathonTestHelper.validateJsonSchemaForString(json, valid = false)
163223 }
164224 }
165225}
View Options

src/test/scala/mesosphere/marathon/api/v2/json/AppDefinitionTest.scala

11package mesosphere.marathon
22package api.v2.json
33
44import com.wix.accord._
55import mesosphere.marathon.api.JsonTestHelper
6import mesosphere.marathon.api.v2.{ AppNormalization, AppsResource, ValidationHelper }
6import mesosphere.marathon.api.v2.{ AppNormalization, AppsResource }
77import mesosphere.marathon.core.health.{ MarathonHttpHealthCheck, MesosCommandHealthCheck, MesosHttpHealthCheck, PortReference }
88import mesosphere.marathon.core.plugin.PluginManager
99import mesosphere.marathon.core.pod.{ BridgeNetwork, ContainerNetwork }
1010import mesosphere.marathon.raml.{ Raml, Resources, SecretDef }
1111import mesosphere.marathon.state.Container.{ Docker, PortMapping }
1212import mesosphere.marathon.state.EnvVarValue._
1313import mesosphere.marathon.state.PathId._
1414import mesosphere.marathon.state._
Show All 16 Lines
3131 private[this] def fromJson(json: String): AppDefinition = {
3232 val raw: raml.App = Json.parse(json).as[raml.App]
3333 Raml.fromRaml(appNormalization(raw))
3434 }
3535
3636 "AppDefinition" should {
3737 "Validation" in {
3838 def shouldViolate(app: AppDefinition, path: String, template: String)(implicit validAppDef: Validator[AppDefinition] = validAppDefinition): Unit = {
39 validate(app) match {
40 case Success => fail(s"expected failure '$template'")
41 case f: Failure =>
42 val violations = ValidationHelper.getAllRuleConstrains(f)
43
44 assert(
45 violations.exists { v =>
46 v.path.contains(path) && v.message == template
47 },
48 s"Violations:\n${violations.mkString}"
49 )
39 validate(app) should containViolation(path, template)
5040 }
51 }
5241
5342 def shouldNotViolate(app: AppDefinition, path: String, template: String)(implicit validAppDef: Validator[AppDefinition] = validAppDefinition): Unit = {
54 validate(app) match {
55 case Success =>
56 case f: Failure =>
57 val violations = ValidationHelper.getAllRuleConstrains(f)
58 assert(
59 !violations.exists { v =>
60 v.path.contains(path) && v.message == template
61 },
62 s"Violations:\n${violations.mkString}"
63 )
43 validate(app) shouldNot containViolation(path, template)
6444 }
65 }
6645
6746 var app = AppDefinition(id = "a b".toRootPath)
6847 val idError = "must fully match regular expression '^(([a-z0-9]|[a-z0-9][a-z0-9\\-]*[a-z0-9])\\.)*([a-z0-9]|[a-z0-9][a-z0-9\\-]*[a-z0-9])|(\\.|\\.\\.)$'"
6948 MarathonTestHelper.validateJsonSchema(app, false)
7049 shouldViolate(app, "/id", idError)
7150
7251 app = app.copy(id = "a#$%^&*b".toRootPath)
7352 MarathonTestHelper.validateJsonSchema(app, false)
7453 shouldViolate(app, "/id", idError)
7554
7655 app = app.copy(id = "-dash-disallowed-at-start".toRootPath)
7756 MarathonTestHelper.validateJsonSchema(app, false)
7857 shouldViolate(app, "/id", idError)
7958
8059 app = app.copy(id = "dash-disallowed-at-end-".toRootPath)
8160 MarathonTestHelper.validateJsonSchema(app, false)
8261 shouldViolate(app, "/id", idError)
8362
8463 app = app.copy(id = "uppercaseLettersNoGood".toRootPath)
8564 MarathonTestHelper.validateJsonSchema(app, false)
8665 shouldViolate(app, "/id", idError)
8766
88 app = AppDefinition(
89 id = "test".toPath,
90 instances = -3,
91 portDefinitions = PortDefinitions(9000, 8080, 9000)
92 )
93 shouldViolate(
94 app,
95 "/portDefinitions",
96 "Ports must be unique."
97 )
98 MarathonTestHelper.validateJsonSchema(app, false)
99
100 app = AppDefinition(
101 id = "test".toPath,
102 portDefinitions = PortDefinitions(0, 0, 8080),
103 cmd = Some("true")
104 )
105 shouldNotViolate(
106 app,
107 "/portDefinitions",
108 "Ports must be unique."
109 )
110 MarathonTestHelper.validateJsonSchema(app, true)
111
112 app = AppDefinition(
113 id = "test".toPath,
114 cmd = Some("true"),
115 networks = Seq(BridgeNetwork()), container = Some(Docker(
116 image = "mesosphere/marathon",
117
118 portMappings = Seq(
119 PortMapping(8080, Some(0), 0, "tcp", Some("foo")),
120 PortMapping(8081, Some(0), 0, "tcp", Some("foo"))
121 )
122 )),
123 portDefinitions = Nil
124 )
125 shouldViolate(
126 app,
127 "/container/portMappings",
128 "Port names must be unique."
129 )
130
131 app = AppDefinition(
132 id = "test".toPath,
133 cmd = Some("true"),
134 portDefinitions = Seq(
135 PortDefinition(port = 9000, name = Some("foo")),
136 PortDefinition(port = 9001, name = Some("foo"))
137 )
138 )
139 shouldViolate(
140 app,
141 "/portDefinitions",
142 "Port names must be unique."
143 )
144
14567 val correct = AppDefinition(id = "test".toRootPath)
146 val containerNetworking = Seq(ContainerNetwork("dcos"))
14768
14869 app = correct.copy(
149 container = Some(Docker(
150 image = "mesosphere/marathon",
151 portMappings = Seq(
152 PortMapping(8080, Some(0), 0, "tcp", Some("foo")),
153 PortMapping(8081, Some(0), 0, "tcp", Some("bar"))
154 )
155 )),
156 portDefinitions = Nil,
157 networks = containerNetworking)
158
159 shouldNotViolate(
160 app,
161 "/container/portMappings",
162 "Port names must be unique."
163 )
164
165 app = correct.copy(
16670 networks = Seq(ContainerNetwork("whatever")), container = Some(Docker(
16771 image = "mesosphere/marathon",
16872
16973 portMappings = Seq(
17074 PortMapping(8080, None, 0, "tcp", Some("foo"))
17175 )
17276 )),
17377 portDefinitions = Nil)
Show All 10 Lines
18488 image = "mesosphere/marathon",
18589 portMappings = Seq(
18690 PortMapping(8080, None, 0, "tcp", Some("foo"))
18791 )
18892 )),
18993 portDefinitions = Nil)
19094 }
19195 caught.getMessage should include("bridge networking requires that every host-port in a port-mapping is non-empty (but may be zero)")
192
193 app = correct.copy(
194 networks = Seq(ContainerNetwork("whatever")), container = Some(Docker(
195 image = "mesosphere/marathon",
196
197 portMappings = Seq(
198 PortMapping(8080, Some(0), 0, "tcp", Some("foo")),
199 PortMapping(8081, Some(0), 0, "tcp", Some("bar"))
200 )
201 )),
202 portDefinitions = Nil)
203 shouldNotViolate(
204 app,
205 "/container/portMappings",
206 "Port names must be unique."
207 )
208
209 // unique port names for USER mode
210 app = correct.copy(
211 networks = Seq(ContainerNetwork("whatever")), container = Some(Docker(
212 image = "mesosphere/marathon",
213
214 portMappings = Seq(
215 PortMapping(8080, Some(0), 0, "tcp", Some("foo")),
216 PortMapping(8081, Some(0), 0, "tcp", Some("foo"))
217 )
218 )),
219 portDefinitions = Nil)
220 shouldViolate(
221 app,
222 "/container/portMappings",
223 "Port names must be unique."
224 )
225
226 app = correct.copy(
227 portDefinitions = Seq(
228 PortDefinition(port = 9000, name = Some("foo")),
229 PortDefinition(port = 9001, name = Some("bar"))
230 )
231 )
232 shouldNotViolate(
233 app,
234 "/portDefinitions",
235 "Port names must be unique."
236 )
23796
23897 app = correct.copy(executor = "//cmd")
23998 shouldNotViolate(
24099 app,
241100 "/executor",
242101 "{javax.validation.constraints.Pattern.message}"
243102 )
244103 MarathonTestHelper.validateJsonSchema(app)
▲ Show 20 Lines • Show All 726 Lines • Show Last 20 Lines
View Options

src/test/scala/mesosphere/marathon/api/v2/validation/AppValidationTest.scala

11package mesosphere.marathon
22package api.v2.validation
33
44import com.wix.accord.scalatest.ResultMatchers
55import mesosphere.{ UnitTest, ValidationTestLike }
6import mesosphere.marathon.raml.{ App, Container, ContainerPortMapping, EngineType, Network, NetworkMode }
6import mesosphere.marathon.raml._
7import mesosphere.UnitTest
78
89class AppValidationTest extends UnitTest with ResultMatchers with ValidationTestLike {
910
1011 import Normalization._
1112
12 "canonical app validation" when {
13 "network validation" when {
14 implicit val basicValidator = AppValidation.validateCanonicalAppAPI(Set.empty)
1315
14 val basicValidator = AppValidation.validateCanonicalAppAPI(Set.empty)
16 def networkedApp(portMappings: Seq[ContainerPortMapping], networks: Seq[Network]) = {
17 App(
18 id = "/foo",
19 cmd = Some("bar"),
20 networks = networks,
21 container = Some(Container(`type` = EngineType.Mesos, portMappings = Some(portMappings))))
22 }
1523
24 def containerNetworkedApp(portMappings: Seq[ContainerPortMapping], networkCount: Int = 1) =
25 networkedApp(
26 portMappings,
27 networks = 1.to(networkCount).map { i => Network(mode = NetworkMode.Container, name = Some(i.toString)) })
28
1629 "multiple container networks are specified for an app" should {
jdefUnsubmitted
Done

combine this and the func below? fakeNetworkedContainer(portMappings: Seq[ContainerPortMapping], networkCount: Int = 1)?

timcharperAuthorUnsubmitted
Done

Reasonable

1730
18 val app = App(id = "/foo", cmd = Some("bar"), networks = 1.to(2).map(i => Network(mode = NetworkMode.Container, name = Some(i.toString))))
31 "require networkNames for hostPort to containerPort mapping" in {
32 val badApp = containerNetworkedApp(
33 Seq(ContainerPortMapping(hostPort = Option(0))), networkCount = 2)
1934
20 // we don't allow this yet because Marathon doesn't yet support per-network port-mapping (and it's not meaningful
21 // for a single host port to map to the same container port on multiple network interfaces).
22 "disallow containerPort to hostPort mapping" in {
23 val ct = Container(`type` = EngineType.Mesos, portMappings = Some(Seq(ContainerPortMapping(hostPort = Option(0)))))
24 val badApp = app.copy(container = Some(ct))
25 basicValidator(badApp).normalize should failWith("/container/portMappings(0)/hostPort" -> "must be empty")
35 basicValidator(badApp).normalize should failWith(
36 "/container/portMappings(0)" ->
37 AppValidationMessages.NetworkNameRequiredForMultipleContainerNetworks)
2638 }
2739
28 "allow portMappings that don't declare hostPort" in {
40 "allow portMappings that don't declare hostPort nor networkNames" in {
jdefUnsubmitted
Done

out of date comment

29 val ct = Container(`type` = EngineType.Mesos, portMappings = Some(Seq(ContainerPortMapping())))
30 val badApp = app.copy(container = Some(ct))
31 basicValidator(badApp) should be(aSuccess)
41 val badApp = containerNetworkedApp(
42 Seq(ContainerPortMapping()), networkCount = 2)
43 basicValidator(badApp) shouldBe (aSuccess)
3244 }
45
jdefUnsubmitted
Done

nice messages!

46 "allow portMappings that both declare a hostPort and a networkNames" in {
47 val badApp = containerNetworkedApp(Seq(
48 ContainerPortMapping(
49 hostPort = Option(0),
50 networkNames = List("1"))), networkCount = 2)
51 basicValidator(badApp) shouldBe (aSuccess)
3352 }
53 }
54
55 "single container network" should {
56
57 "consider a valid portMapping with a name as valid" in {
58 basicValidator(
59 containerNetworkedApp(
60 Seq(
61 ContainerPortMapping(
62 hostPort = Some(80),
63 containerPort = 80,
64 networkNames = List("1"))))) shouldBe (aSuccess)
65 }
66
67 "consider a portMapping with a hostPort and two valid networkNames as invalid" in {
68 basicValidator(
jdefUnsubmitted
Done

nit: why Success vs aSuccess?

timcharperAuthorUnsubmitted
Not Done

¯\_(ツ)_/¯ why are there two ways to do it?

69 containerNetworkedApp(
70 Seq(
71 ContainerPortMapping(
72 hostPort = Some(80),
73 containerPort = 80,
74 networkNames = List("1", "2"))),
75 networkCount = 3)) should containViolation(
76 "/container/portMappings(0)" -> AppValidationMessages.NetworkNameRequiredForMultipleContainerNetworks)
77 }
78
79 "consider a portMapping with no name as valid" in {
80 basicValidator(
81 containerNetworkedApp(
jdefUnsubmitted
Done

s/maybe//

82 Seq(
83 ContainerPortMapping(
84 hostPort = Some(80),
85 containerPort = 80,
86 networkNames = Nil)))) shouldBe (aSuccess)
87 }
88
89 "consider a portMapping without a hostport as valid" in {
90 basicValidator(
91 containerNetworkedApp(
92 Seq(
93 ContainerPortMapping(
94 hostPort = None)))) shouldBe (aSuccess)
95 }
96
97 "consider portMapping with zero hostport as valid" in {
98 basicValidator(
jdefUnsubmitted
Done

redundant?

timcharperAuthorUnsubmitted
Done

good catch

99 containerNetworkedApp(
100 Seq(
101 ContainerPortMapping(
102 containerPort = 80,
103 hostPort = Some(0))))) shouldBe (aSuccess)
104 }
105
106 "consider portMapping with a non-matching network name as invalid" in {
107 val result = basicValidator(
108 containerNetworkedApp(
109 Seq(
110 ContainerPortMapping(
111 containerPort = 80,
112 hostPort = Some(80),
113 networkNames = List("undefined-network-name")))))
114 result.isFailure shouldBe true
115 }
jdefUnsubmitted
Done

nit: s/invalid/undefined/

116
117 "consider portMapping without networkNames nor hostPort as valid" in {
118 basicValidator(
119 containerNetworkedApp(
120 Seq(
121 ContainerPortMapping(
122 containerPort = 80,
123 hostPort = None,
124 networkNames = Nil)))) shouldBe (aSuccess)
125 }
126 }
127
128 "general port validation" in {
129 basicValidator(
130 containerNetworkedApp(
131 Seq(
132 ContainerPortMapping(
133 name = Some("name"),
134 hostPort = Some(123)),
135 ContainerPortMapping(
136 name = Some("name"),
137 hostPort = Some(123))))).isFailure shouldBe true
138 }
139
140 "missing hostPort is allowed for bridge networking (so we can normalize it)" in {
141 // This isn't _actually_ allowed; we expect that normalization will replace the None to a Some(0) before
142 // converting to an AppDefinition, in order to support legacy API
143 val app = networkedApp(
144 portMappings = Seq(ContainerPortMapping(
145 containerPort = 8080,
146 hostPort = None,
147 servicePort = 0,
148 name = Some("foo"))),
149 networks = Seq(Network(mode = NetworkMode.ContainerBridge, name = None))
150 )
151 basicValidator(app) shouldBe (aSuccess)
152 }
153
34154 }
35155}
View Options

src/test/scala/mesosphere/marathon/api/v2/validation/PodsValidationTest.scala

11package mesosphere.marathon
22package api.v2.validation
33
44import com.wix.accord.Validator
55import com.wix.accord.scalatest.ResultMatchers
6import mesosphere.UnitTest
6import mesosphere.{ UnitTest, ValidationTestLike }
77import mesosphere.marathon.raml.{ Constraint, ConstraintOperator, Endpoint, Network, NetworkMode, Pod, PodContainer, Resources, Volume, VolumeMount }
88import mesosphere.marathon.util.SemanticVersion
99
10class PodsValidationTest extends UnitTest with ResultMatchers with PodsValidation with SchedulingValidation {
10class PodsValidationTest extends UnitTest with ResultMatchers with PodsValidation with SchedulingValidation with ValidationTestLike {
1111
1212 "A pod definition" should {
1313
1414 "be rejected if the id is empty" in new Fixture {
1515 private val invalid = validPod.copy(id = "/")
1616 validator(invalid) should failWith("id" -> "Path must contain at least one path element")
1717 }
1818
Show All 9 Lines
2828
2929 "be rejected if no container is defined" in new Fixture {
3030 private val invalid = validPod.copy(containers = Seq.empty)
3131 validator(invalid) should failWith("containers" -> "must not be empty")
3232 }
3333
3434 "be rejected if container names are not unique" in new Fixture {
3535 private val invalid = validPod.copy(containers = Seq(validContainer, validContainer))
36 validator(invalid) should failWith("containers" -> "container names are unique")
36 validator(invalid) should failWith("containers" -> PodsValidationMessages.ContainerNamesMustBeUnique)
3737 }
3838
3939 "be rejected if endpoint names are not unique" in new Fixture {
4040 val endpoint1 = Endpoint("endpoint", hostPort = Some(123))
4141 val endpoint2 = Endpoint("endpoint", hostPort = Some(124))
4242 private val invalid = validPod.copy(containers = Seq(validContainer.copy(endpoints = Seq(endpoint1, endpoint2))))
43 validator(invalid) should failWith("value" -> "Endpoint names are unique")
43 validator(invalid) should failWith("value" -> PodsValidationMessages.EndpointNamesMustBeUnique)
4444 }
4545
4646 "be rejected if endpoint host ports are not unique" in new Fixture {
4747 val endpoint1 = Endpoint("endpoint1", hostPort = Some(123))
4848 val endpoint2 = Endpoint("endpoint2", hostPort = Some(123))
4949 private val invalid = validPod.copy(containers = Seq(validContainer.copy(endpoints = Seq(endpoint1, endpoint2))))
50 validator(invalid) should failWith("value" -> "Host ports are unique")
50 validator(invalid) should failWith("value" -> PodsValidationMessages.HostPortsMustBeUnique)
5151 }
5252
5353 "be rejected if endpoint container ports are not unique" in new Fixture {
5454 val endpoint1 = Endpoint("endpoint1", containerPort = Some(123))
5555 val endpoint2 = Endpoint("endpoint2", containerPort = Some(123))
5656 private val invalid = validPod.copy(
5757 networks = Seq(Network(mode = NetworkMode.Container)),
5858 containers = Seq(validContainer.copy(endpoints = Seq(endpoint1, endpoint2)))
5959 )
60 validator(invalid) should failWith("value" -> "Container ports are unique")
60 validator(invalid) should failWith("value" -> PodsValidationMessages.ContainerPortsMustBeUnique)
6161 }
6262
6363 "be rejected if volume names are not unique" in new Fixture {
6464 val volume = Volume("volume", host = Some("/foo"))
6565 val volumeMount = VolumeMount(volume.name, "/bla")
6666 private val invalid = validPod.copy(
6767 volumes = Seq(volume, volume),
6868 containers = Seq(validContainer.copy(volumeMounts = Seq(volumeMount)))
6969 )
70 validator(invalid) should failWith("volumes" -> "volume names are unique")
70 validator(invalid) should failWith("volumes" -> PodsValidationMessages.VolumeNamesMustBeUnique)
7171 }
7272 }
7373
7474 "A constraint definition" should {
7575
7676 "MaxPer is accepted with an integer value" in {
7777 complyWithConstraintRules(Constraint("foo", ConstraintOperator.MaxPer, Some("3"))).isSuccess shouldBe true
7878 }
Show All 9 Lines
8888 resources = Resources()
8989 )
9090 val validPod = Pod(
9191 id = "/some/pod",
9292 containers = Seq(validContainer),
9393 networks = Seq(Network(mode = NetworkMode.Host))
9494 )
9595 val validator: Validator[Pod] = podDefValidator(Set.empty, SemanticVersion.zero)
96 }
97
98 "network validation" when {
99 val validator: Validator[Pod] = podDefValidator(Set.empty, SemanticVersion.zero)
100
101 def podContainer(name: String = "ct1", resources: Resources = Resources(), endpoints: Seq[Endpoint] = Nil) =
102 PodContainer(
103 name = name,
104 resources = resources,
105 endpoints = endpoints)
106
107 def containerNetworkedPod(containers: Seq[PodContainer], networkCount: Int = 1) =
jdefUnsubmitted
Not Done

confusingly named func. suggest containerNetworkedPod since there are actually different validation rules for bridge vs container networks

timcharperAuthorUnsubmitted
Not Done

thanks, I had fixed this in the other suite but missed this

108 Pod(
109 id = "/foo",
110 networks = 1.to(networkCount).map(i => Network(mode = NetworkMode.Container, name = Some(i.toString))),
111 containers = containers)
112
113 "multiple container networks are specified for a pod" should {
114
115 "require networkNames for containerPort to hostPort mapping" in {
jdefUnsubmitted
Done

this comment is confusing because this changeset is actually adding suport for per-network port-mapping

116 val badApp = containerNetworkedPod(
117 Seq(podContainer(endpoints = Seq(Endpoint("endpoint", containerPort = Some(80), hostPort = Option(0))))),
118 networkCount = 2)
119
120 validator(badApp).isFailure shouldBe true
121 }
122
123 "allow portMappings that don't declare hostPort nor networkNames" in {
124 val app = containerNetworkedPod(
125 Seq(podContainer(endpoints = Seq(Endpoint("endpoint", containerPort = Some(80))))),
126 networkCount = 2)
127 validator(app) shouldBe (aSuccess)
128 }
129
130 "allow portMappings that both declare a hostPort and a networkNames" in {
131 val app = containerNetworkedPod(
132 Seq(podContainer(endpoints = Seq(
133 Endpoint(
134 "endpoint",
135 hostPort = Option(0),
136 containerPort = Some(80),
137 networkNames = List("1"))))),
138 networkCount = 2)
139 validator(app) shouldBe (aSuccess)
140 }
141 }
jdefUnsubmitted
Not Done

would also like to see "single-item networkNames list mandatory" and containerPort=Some(_), hostPort=Some(_), networkNames=List("1","2")

142
143 "single container network" should {
144
145 "consider a valid portMapping with a networkNames as valid" in {
146 validator(
147 containerNetworkedPod(Seq(
jdefUnsubmitted
Done

it's late, and my eyes can't find where this is used, other than in the decl below

148 podContainer(endpoints = Seq(
jdefUnsubmitted
Done

how is this portMappingValidator ever used to validate endpoints?

timcharperAuthorUnsubmitted
Not Done

it's not; left over in the long chain of changes. thanks for catching

149 Endpoint(
150 "endpoint",
151 hostPort = Some(80),
jdefUnsubmitted
Done

s/name/networkName/ (name is actually a different field)

152 containerPort = Some(80),
153 networkNames = List("1"))))))) shouldBe (aSuccess)
154 }
155
156 "consider a portMapping with a host port and two valid networkNames as invalid" in {
157 validator(
158 containerNetworkedPod(Seq(
159 podContainer(endpoints = Seq(
160 Endpoint(
161 "endpoint",
162 hostPort = Some(80),
jdefUnsubmitted
Done

ditto

163 containerPort = Some(80),
164 networkNames = List("1", "2"))))))) shouldBe (aFailure)
165 }
166
167 "consider a portMapping with no networkNames as valid" in {
168 validator(
169 containerNetworkedPod(Seq(
170 podContainer(endpoints = Seq(
171 Endpoint(
172 "endpoint",
173 hostPort = Some(80),
174 containerPort = Some(80),
175 networkNames = Nil)))))) shouldBe (aSuccess)
176 }
177
178 "maybe consider a portMapping without hostport as valid" in {
179 validator(
180 containerNetworkedPod(Seq(
181 podContainer(endpoints = Seq(
182 Endpoint(
183 "endpoint",
184 hostPort = None,
185 containerPort = Some(80),
186 networkNames = Nil)))))) shouldBe (aSuccess)
187 }
188
189 "consider portMapping with zero hostport as valid" in {
190 validator(
191 containerNetworkedPod(Seq(
192 podContainer(endpoints = Seq(
193 Endpoint(
194 "endpoint",
195 containerPort = Some(80),
196 hostPort = Some(0))))))) shouldBe (aSuccess)
197 }
198
199 "consider portMapping with a non-matching network name as invalid" in {
200 val result = validator(
201 containerNetworkedPod(Seq(
202 podContainer(endpoints = Seq(
203 Endpoint(
204 "endpoint",
205 containerPort = Some(80),
206 hostPort = Some(80),
207 networkNames = List("invalid-network-name")))))))
208 result.isFailure shouldBe true
209 }
210
211 "consider portMapping without networkNames nor hostPort as valid" in {
212 validator(
213 containerNetworkedPod(Seq(
214 podContainer(endpoints = Seq(
215 Endpoint(
216 "endpoint",
217 containerPort = Some(80),
218 hostPort = None,
219 networkNames = Nil)))))) shouldBe (aSuccess)
220 }
221 }
222
223 "require that hostPort is unique" in {
224 val result = validator(
225 containerNetworkedPod(Seq(
226 podContainer(endpoints = Seq(
227 Endpoint(
228 "name",
229 hostPort = Some(123)),
230 Endpoint(
231 "name",
232 hostPort = Some(123)))))))
233
234 result should containViolation(
235 "/containers(0)/endpoints(0)/containerPort" -> "is required when using container-mode networking")
236 result should containViolation(
237 "/" -> PodsValidationMessages.EndpointNamesMustBeUnique)
238 result should containViolation(
239 "/" -> PodsValidationMessages.HostPortsMustBeUnique)
240 }
96241 }
97242}
View Options

src/test/scala/mesosphere/marathon/raml/NetworkConversionTest.scala

Show All 30 Lines
3131 val raml = portDefinition.toRaml[PortDefinition]
3232 raml.port should be(portDefinition.port)
3333 raml.name should be(portDefinition.name)
3434 raml.labels should be(portDefinition.labels)
3535 raml.protocol should be(NetworkProtocol.Udp)
3636 }
3737 }
3838 "NetworkConversion port mapping conversion" should {
39 "convert correctly" in {
40 val portMapping = PortMapping(23, Some(123), 0, "udp", Some("name"), Map("foo" -> "bla"))
41 val raml = portMapping.toRaml[ContainerPortMapping]
42 raml.containerPort should be(portMapping.containerPort)
43 raml.hostPort should be(portMapping.hostPort)
44 raml.servicePort should be(portMapping.servicePort)
45 raml.name should be(portMapping.name)
46 raml.labels should be(portMapping.labels)
47 raml.protocol should be(NetworkProtocol.Udp)
39 "convert between model and raml" in {
40 val portMapping = PortMapping(23, Some(123), 0, "udp", Some("name"), Map("foo" -> "bla"), List("network-name"))
41 val roundTripped = portMapping.toRaml[ContainerPortMapping].fromRaml[PortMapping]
42 roundTripped shouldBe portMapping
43 }
44
45 "convert between protobuf and raml" in {
46 val b = Protos.ExtendedContainerInfo.PortMapping.newBuilder
47 .setName("port-jr")
48 .addNetworkNames("network-1")
49 .addNetworkNames("network-2")
50 .setHostPort(123)
51 .setContainerPort(456)
52 .setProtocol("udp")
53 .setServicePort(999)
54
55 b.addLabelsBuilder.setKey("business-level").setValue("serious")
56
57 val raml = b.build.toRaml
58
59 raml shouldBe ContainerPortMapping(
60 name = Some("port-jr"),
61 hostPort = Some(123),
62 containerPort = 456,
63 protocol = NetworkProtocol.Udp,
64 servicePort = 999,
65 labels = Map("business-level" -> "serious"),
66 networkNames = List("network-1", "network-2")
67 )
4868 }
4969 }
5070 "NetworkConversion network conversion" should {
5171 behave like convertToProtobufThenToRAML(
5272 "host network",
5373 core.pod.HostNetwork, Network(mode = NetworkMode.Host))
5474 behave like convertToProtobufThenToRAML(
5575 "container network named 'foo'",
5676 core.pod.ContainerNetwork("foo", Map("qwe" -> "asd")),
5777 Network(name = Option("foo"), mode = NetworkMode.Container, labels = Map("qwe" -> "asd")))
5878 behave like convertToProtobufThenToRAML(
5979 "bridge network",
6080 core.pod.BridgeNetwork(Map("qwe" -> "asd")),
6181 Network(mode = NetworkMode.ContainerBridge, labels = Map("qwe" -> "asd")))
6282 }
6383}
View Options

src/test/scala/mesosphere/marathon/state/AppDefinitionTest.scala

Show First 20 Lines • Show All 127 Lines • ▼ Show 20 Line(s)
128128 val proto = app.toProto
129129 proto.getId should be("app-with-ip-address")
130130 assert(proto.getNetworksCount > 0)
131131
132132 val read = AppDefinition(id = runSpecId).mergeFromProto(proto)
133133 read should be(app)
134134 }
135135
136 "ipAddress to proto and back again w/ Docker container w/ virtual networking" in {
136 "app to proto and back again w/ Docker container w/ virtual networking" in {
137137 val app = AppDefinition(
138 id = "app-with-ip-address".toPath,
138 id = "app-with-port-mappings".toPath,
139139 cmd = Some("sleep 30"),
140140 portDefinitions = Nil,
141 networks = Seq(ContainerNetwork(
141 networks = Seq(
142 ContainerNetwork(
142143 labels = Map(
143144 "foo" -> "bar",
144145 "baz" -> "buzz"
145146 ),
146147 name = "blahze"
147148 )),
148149
149150 container = Some(Container.Docker(
150151 image = "jdef/foo",
151152
152153 portMappings = Seq(
153154 Container.PortMapping(hostPort = None),
154155 Container.PortMapping(hostPort = Some(123)),
155 Container.PortMapping(containerPort = 1, hostPort = Some(234), protocol = "udp")
156 Container.PortMapping(
157 containerPort = 1, hostPort = Some(234), protocol = "udp", networkNames = List("blahze"))
156158 )
157159 ))
158160 )
159161
160162 val proto = app.toProto
161 proto.getId should be("app-with-ip-address")
163 proto.getId should be("app-with-port-mappings")
162164 assert(proto.getNetworksCount > 0)
163165
164166 val read = AppDefinition(id = runSpecId).mergeFromProto(proto)
165167 read should be(app)
166168 }
167169
168170 "ipAddress to proto and back again w/ Docker container w/ bridge" in {
169171 val app = AppDefinition(
▲ Show 20 Lines • Show All 143 Lines • Show Last 20 Lines
View Options

src/test/scala/mesosphere/marathon/state/PortMappingTest.scala

  • This file was deleted.
This file was completely deleted. Show File Contents
View Options

src/test/scala/mesosphere/mesos/PortDiscoveryTest.scala

  • This file was added.
1package mesosphere.mesos
2
3import mesosphere.UnitTest
4import mesosphere.marathon.core.pod.{ BridgeNetwork, ContainerNetwork, HostNetwork }
5import mesosphere.marathon.raml.Endpoint
6import mesosphere.marathon.state.Container.{ Mesos, PortMapping }
7import mesosphere.marathon.state.{ AppDefinition, PathId, PortDefinition }
8
9import scala.collection.immutable.Seq
10import mesosphere.mesos.protos.Implicits._
11
12class PortDiscoveryTest extends UnitTest {
13 def containerNetworks(qty: Int = 1) = (1 to qty).map{ i => ContainerNetwork(s"network-${i.toString}") }
14
15 "generating for pods" should {
16 "generate a network-name label for endpoints specifying a network name and not a host port" in {
jdefUnsubmitted
Done

maybe a test for a container-scope DI that doesn't generate a network-name label?

timcharperAuthorUnsubmitted
Done

with the changes we discussed today, will this ever happen? (not specifying networkName == specify all the networks)

jdefUnsubmitted
Done

no change needed if we implement what was agreed upon today via hangouts.

17 val Seq(discovery) = PortDiscovery.generateForPod(
18 containerNetworks(1),
19 endpoints = Seq(
20 Endpoint(
21 name = "service",
22 containerPort = Some(80),
23 hostPort = None,
24 protocol = List("lolocol"),
25 networkNames = List("network-1"))))
26
27 discovery.getName shouldBe "service"
28 discovery.getProtocol shouldBe "lolocol"
29 discovery.getLabels.fromProto shouldBe (
30 Map(
31 PortDiscovery.NetworkScopeLabel -> PortDiscovery.NetworkScopeContainer,
32 PortDiscovery.NetworkNameLabel -> "network-1"))
33 }
34
35 "not generate a network-name label for endpoints specifying a host port" in {
36 val Seq(discovery) = PortDiscovery.generateForPod(
37 containerNetworks(1),
38 endpoints = Seq(
39 Endpoint(
40 name = "service",
41 containerPort = Some(80),
42 hostPort = Some(1000),
43 protocol = List("lolocol"),
44 networkNames = List("network-1"))))
45
46 discovery.getLabels.fromProto shouldBe (
47 Map(
48 PortDiscovery.NetworkScopeLabel -> PortDiscovery.NetworkScopeHost))
49 }
50
51 "generate a label for all networks when networkNames is Nil" in {
52 val discovery = PortDiscovery.generateForPod(
53 containerNetworks(2),
54 endpoints = Seq(
55 Endpoint(
56 name = "service",
57 containerPort = Some(80),
58 protocol = List("lolocol"),
59 networkNames = Nil)))
60
61 discovery.map(_.getLabels.fromProto) shouldBe Seq(
62 Map(
63 PortDiscovery.NetworkScopeLabel -> PortDiscovery.NetworkScopeContainer,
64 PortDiscovery.NetworkNameLabel -> "network-1"),
65 Map(
66 PortDiscovery.NetworkScopeLabel -> PortDiscovery.NetworkScopeContainer,
67 PortDiscovery.NetworkNameLabel -> "network-2"))
68 }
69
70 "generate a label for only the specified networkNames" in {
71 val discovery = PortDiscovery.generateForPod(
72 containerNetworks(3),
73 endpoints = Seq(
74 Endpoint(
75 name = "service",
76 containerPort = Some(80),
77 protocol = List("lolocol"),
78 networkNames = List("network-1", "network-2"))))
79
80 discovery.map(_.getLabels.fromProto) shouldBe Seq(
jdefUnsubmitted
Done

good example of a bad model being used for a test. should be impossible for hostPort to be None here while the host ports list passed to generateForApp contains an allocated host port. see earlier comment about adding a catch-all case statement to flag things like this

timcharperAuthorUnsubmitted
Done

lol, I think this might be irony if I had argued more strongly against require statements in models :)

81 Map(
82 PortDiscovery.NetworkScopeLabel -> PortDiscovery.NetworkScopeContainer,
83 PortDiscovery.NetworkNameLabel -> "network-1"),
84 Map(
85 PortDiscovery.NetworkScopeLabel -> PortDiscovery.NetworkScopeContainer,
86 PortDiscovery.NetworkNameLabel -> "network-2"))
87 }
88 }
89
90 "generating for apps" when {
91 "using container networks" should {
92 "generate a network-name label for mappings specifying a network name and not a host port" in {
93 val app = AppDefinition(
94 PathId("/test"),
95 networks = containerNetworks(1),
96 container = Some(Mesos(
97 portMappings = Seq(PortMapping(
98 name = Some("service"),
99 containerPort = 80,
100 hostPort = None,
101 networkNames = List("network-1"))))))
102
103 val Seq(discovery) = PortDiscovery.generateForApp(app, Seq(None))
104 discovery.getLabels.fromProto shouldBe (
105 Map(
106 PortDiscovery.NetworkScopeLabel -> PortDiscovery.NetworkScopeContainer,
107 PortDiscovery.NetworkNameLabel -> "network-1"))
108 }
109
110 "not generate a network-name label for mappings with a host port" in {
111 val app = AppDefinition(
112 PathId("/test"),
113 networks = containerNetworks(1),
114 container = Some(Mesos(
115 portMappings = Seq(PortMapping(
116 name = Some("service"),
117 containerPort = 80,
118 hostPort = Some(0),
119 networkNames = List("network-1"))))))
120
121 val Seq(discovery) = PortDiscovery.generateForApp(app, Seq(Some(31500)))
122 discovery.getLabels.fromProto shouldBe (
123 Map(
124 PortDiscovery.NetworkScopeLabel -> PortDiscovery.NetworkScopeHost))
125 }
126
127 "generate a label for all networks when networkNames is Nil" in {
128 val app = AppDefinition(
129 PathId("/test"),
130 networks = containerNetworks(2),
131 container = Some(Mesos(
132 portMappings = Seq(PortMapping(
133 name = Some("service"),
134 containerPort = 80,
135 hostPort = None,
136 networkNames = Nil)))))
137 val discovery = PortDiscovery.generateForApp(app, Seq(None))
138
139 discovery.map(_.getLabels.fromProto) shouldBe Seq(
140 Map(
141 PortDiscovery.NetworkScopeLabel -> PortDiscovery.NetworkScopeContainer,
142 PortDiscovery.NetworkNameLabel -> "network-1"),
143 Map(
144 PortDiscovery.NetworkScopeLabel -> PortDiscovery.NetworkScopeContainer,
145 PortDiscovery.NetworkNameLabel -> "network-2"))
146 }
147
148 "generate a label for only the specified networkNames" in {
149 val app = AppDefinition(
150 PathId("/test"),
151 networks = containerNetworks(3),
152 container = Some(Mesos(
153 portMappings = Seq(PortMapping(
154 name = Some("service"),
155 containerPort = 80,
156 hostPort = None,
157 networkNames = List("network-1", "network-2"))))))
158
159 val discovery = PortDiscovery.generateForApp(app, Seq(None))
160
161 discovery.map(_.getLabels.fromProto) shouldBe Seq(
162 Map(
163 PortDiscovery.NetworkScopeLabel -> PortDiscovery.NetworkScopeContainer,
164 PortDiscovery.NetworkNameLabel -> "network-1"),
165 Map(
166 PortDiscovery.NetworkScopeLabel -> PortDiscovery.NetworkScopeContainer,
167 PortDiscovery.NetworkNameLabel -> "network-2"))
168 }
169
170 "generate a record for each specified protocol" in {
jdefUnsubmitted
Not Done

nice. can we have a test for app w/ host networking (that uses PortDefinition)? I suspect adding one (without changing any of the DI generation code) will fail because of the missing support for PortDefinition

171 val app = AppDefinition(
172 PathId("/test"),
173 networks = containerNetworks(3),
174 container = Some(Mesos(
175 portMappings = Seq(PortMapping(
176 name = Some("service"),
177 containerPort = 80,
178 hostPort = None,
179 protocol = PortMapping.UDP_TCP,
180 networkNames = List("network-1"))))))
181
182 val discovery = PortDiscovery.generateForApp(app, Seq(None))
183
184 discovery.map(_.getProtocol).toSet shouldBe Set(PortMapping.UDP, PortMapping.TCP)
185 }
186 }
187
188 "using bridge networking" should {
189 "generate host DiscoveryInfo records" in {
190 val app = AppDefinition(
191 PathId("/test"),
192 networks = Seq(BridgeNetwork()),
193 container = Some(Mesos(
194 portMappings = Seq(PortMapping(
195 name = Some("service"),
196 containerPort = 80,
197 hostPort = Some(0),
198 protocol = PortMapping.TCP)))))
199
200 val Seq(discovery) = PortDiscovery.generateForApp(app, Seq(Some(1000)))
201 discovery.getNumber shouldBe 1000
202 discovery.getProtocol shouldBe PortMapping.TCP
203 discovery.getName shouldBe "service"
204 discovery.getLabels.fromProto shouldBe Map("network-scope" -> "host")
205 }
206 }
207
208 "using host networking" should {
209 "generate host DiscoveryInfo records for all defined protocols" in {
210 val labels = Map("VIP" -> "127.0.0.1:8080")
211 val app = AppDefinition(
212 PathId("/test"),
213 networks = Seq(HostNetwork),
214 portDefinitions = Seq(
215 PortDefinition(0, "tcp,udp", Some("http"), labels)))
216
217 val Seq(result1, result2) = PortDiscovery.generateForApp(app, List(Some(8080)))
218 result1.getNumber shouldBe (8080)
219 result1.getLabels.fromProto shouldBe labels
220 result1.getName shouldBe "http"
221 result1.getProtocol shouldBe "tcp"
222 result2.getProtocol shouldBe "udp"
223 }
224 }
225 }
226}
View Options

src/test/scala/mesosphere/mesos/TaskBuilderTest.scala

Show First 20 Lines • Show All 278 Lines • ▼ Show 20 Line(s)
279279 val (taskInfo, networkInfo) = task.get
280280
281281 val discoveryInfo = taskInfo.getDiscovery
282282 val discoveryInfoProto = MesosProtos.DiscoveryInfo.newBuilder
283283 .setVisibility(MesosProtos.DiscoveryInfo.Visibility.FRAMEWORK)
284284 .setName(taskInfo.getName)
285285 .setPorts(Helpers.mesosPorts(
286286 Helpers.mesosPort("http", "tcp", Map("VIP" -> "127.0.0.1:8080", "network-scope" -> "host"), networkInfo.hostPorts.head),
287 Helpers.mesosPort("admin", "udp", Map("VIP" -> "127.0.0.1:8081", "network-scope" -> "container"), 8081)
287 Helpers.mesosPort("admin", "udp", Map("VIP" -> "127.0.0.1:8081", "network-scope" -> "container", "network-name" -> "whatever"), 8081)
288288 )).build
289289
290290 TextFormat.shortDebugString(discoveryInfo) should equal(TextFormat.shortDebugString(discoveryInfoProto))
291291 discoveryInfo should equal(discoveryInfoProto)
292292 }
293293
294294 "BuildIfMatches works with duplicated resources" in {
295295 val offer = MarathonTestHelper.makeBasicOffer(cpus = 1.0, mem = 128.0, disk = 2000.0, beginPort = 31000, endPort = 32000)
▲ Show 20 Lines • Show All 623 Lines • ▼ Show 20 Line(s)
919919 .setName(taskInfo.getName)
920920 .setPorts(
921921 MesosProtos.Ports.newBuilder
922922 .addPorts(
923923 MesosProtos.Port.newBuilder
924924 .setName("http")
925925 .setNumber(80)
926926 .setProtocol("tcp")
927 .setLabels(Map("network-scope" -> "container").toMesosLabels)
927 .setLabels(Map("network-scope" -> "container", "network-name" -> "whatever").toMesosLabels)
928928 .build)
929929 .build)
930930 .build
931931 TextFormat.printToString(discoveryInfo) should equal(TextFormat.printToString(discoveryInfoProto))
932932 discoveryInfo should equal(discoveryInfoProto)
933933 }
934934
935935 "BuildIfMatchesWithCommandAndExecutor" in {
▲ Show 20 Lines • Show All 992 Lines • Show Last 20 Lines
View Options

src/test/scala/mesosphere/mesos/TaskGroupBuilderTest.scala

11package mesosphere.mesos
22
33import mesosphere.UnitTest
44import mesosphere.marathon.core.health.{ MesosCommandHealthCheck, MesosHttpHealthCheck, MesosTcpHealthCheck, PortReference }
55import mesosphere.marathon.core.instance.Instance
66import mesosphere.marathon.core.pod._
77import mesosphere.marathon.core.task.Task
88import mesosphere.marathon.plugin.task.RunSpecTaskProcessor
99import mesosphere.marathon.plugin.{ ApplicationSpec, PodSpec }
1010import mesosphere.marathon.raml
11import mesosphere.marathon.raml.Resources
11import mesosphere.marathon.raml.{ Resources, Endpoint }
1212import mesosphere.marathon.state.PathId._
1313import mesosphere.marathon.state.{ Command, EnvVarString, ResourceRole }
1414import mesosphere.marathon.stream.Implicits._
1515import mesosphere.marathon.test.MarathonTestHelper
1616import org.apache.mesos.Protos.{ ExecutorInfo, TaskGroupInfo, TaskInfo }
1717import org.apache.mesos.{ Protos => mesos }
18import org.scalatest.Inside
1819
1920import scala.collection.immutable.Seq
21import scala.collection.JavaConverters._
2022import scala.collection.breakOut
2123
22class TaskGroupBuilderTest extends UnitTest {
24class TaskGroupBuilderTest extends UnitTest with Inside {
2325 val defaultBuilderConfig = TaskGroupBuilder.BuilderConfig(
2426 acceptedResourceRoles = Set(ResourceRole.Unreserved),
2527 envVarsPrefix = None)
2628
2729 "A TaskGroupBuilder" must {
30
31 "correlate Endpoint with the appropriate network" when {
32 "multiple container networks are defined" in {
33
34 val Seq(network1, network2) = TaskGroupBuilder.buildMesosNetworks(
35 List(ContainerNetwork("1"), ContainerNetwork("2")),
36 List(
37 Endpoint("port-1", containerPort = Some(2001), networkNames = List("1")),
38 Endpoint("port-2", containerPort = Some(2002), networkNames = List("2"))),
39 List(
40 Some(1001),
41 Some(1002)))
42
43 network1.getName shouldBe "1"
44 inside(network1.getPortMappingsList.asScala.toList) {
45 case List(port1) =>
46 port1.getContainerPort shouldBe 2001
47 port1.getHostPort shouldBe 1001
48 }
49 inside(network2.getPortMappingsList.asScala.toList) {
50 case List(port2) =>
51 port2.getContainerPort shouldBe 2002
52 port2.getHostPort shouldBe 1002
53 }
54 }
55 }
56
2857 "build from a PodDefinition with a single container" in {
2958 val offer = MarathonTestHelper.makeBasicOffer(cpus = 1.1, mem = 160.0, disk = 10.0).build
3059
3160 val podSpec = PodDefinition(
3261 id = "/product/frontend".toPath,
3362 containers = Seq(
3463 MesosContainer(
3564 name = "Foo",
▲ Show 20 Lines • Show All 856 Lines • Show Last 20 Lines