HomeMesosphereNo notifications. 4 unresolved issues.
  • Search
Differential D793

Write documentation for backup & restore.
ClosedAll Users
Actions

Authored by jeschkies on May 17 2017, 1:50 PM.

Details

Reviewers
sascala
jenkins
aquamatthias
Commits
rMARATHON03eba4c43991: Write documentation for backup & restore.
rMARATHON5330844f30e0: Write documentation for backup & restore.
rMARATHON82ead24615f3: Documentation for backup & Restore (#5397)
JIRA Issues
JIRA MARATHON-2040 Write a gh-pages site for backup & restore
Test Plan

read

Diff Detail

Repository
rMARATHON marathon
Lint
Automatic diff as part of commit; lint not applicable.
Unit
Automatic diff as part of commit; unit tests not applicable.
Changes from before your most recent comment are hidden. Show Older Changes
jeschkies commandeered this revision.May 17 2017, 5:48 PM
jeschkies edited reviewers, added: aquamatthias; removed: jeschkies.

Taking over :)

jenkins requested changes to this revision.May 17 2017, 5:49 PM
This revision now requires changes to proceed.May 17 2017, 5:49 PM
This revision is now accepted and ready to land.May 18 2017, 12:05 AM
Closed by commit rMARATHON82ead24615f3: Documentation for backup & Restore (#5397) (authored by aquamatthias, committed by sascala). · Explain WhyMay 19 2017, 10:24 AM
This revision was automatically updated to reflect the committed changes.

Revision Contents

PathPackages
Mdocs/_layouts/docs.html (5 lines)
AMdocs/docs/backup-restore.md (118 lines)

Diff 3278

View Options

docs/_layouts/docs.html

Show First 20 Lines • Show All 125 Lines • ▼ Show 20 Line(s)
126126 <h5>Administration</h5>
127127 <ul class="list-unstyled">
128128 <li>
129129 <a href="{{ site.baseurl }}/docs/command-line-flags.html">
130130 Command Line Flags
131131 </a>
132132 </li>
133133 <li>
134 <a href="{{ site.baseurl }}/docs/backup-restore.html">
135 Backup and Restore
136 </a>
137 </li>
138 <li>
134139 <a href="{{ site.baseurl }}/docs/framework-authentication.html">
135140 Framework Authentication
136141 </a>
137142 </li>
138143 <li>
139144 <a href="{{ site.baseurl }}/docs/high-availability.html">
140145 High Availability
141146 </a>
▲ Show 20 Lines • Show All 98 Lines • Show Last 20 Lines
View Options

docs/docs/backup-restore.md

  • This file was added.
1---
2title: Backup & Restore
3---
4
5# Backup and Restore
6
7As of version 1.5, Marathon has built-in backup and restore functionality.
8The complete state of Marathon, which is kept in the persistent data store,
jeschkiesAuthorUnsubmitted

Should it be built in? I'm not so sure.

9can be backed up to an external file or to an external storage provider.
10Restoring from a backup brings Marathon to the exact state it was in at the time of backup creation.
11
12## Limitations
13
14Marathon backups only capture the state of Marathon itself.
15A Marathon backup will **not** capture the state of any other system component.
16To create a full backup of your cluster, you must back up all system components.
17
18Restoring a backup in Marathon without restoring state in other system components can lead to unwanted behavior.
19Tasks that Marathon has started after the backup was created will be unknown to Marathon after a restore operation and will be killed.
20Tasks that Marathon has killed after the backup was created will be expunged during the next reconciliation.
21All changes to apps, groups, and pods after the backup has been created, are lost if this backup is restored.
22
23## Backup
24
25A backup operation reads the existing state from Marathon in ZooKeeper and writes this data to a storage location.
26This data contains all entities under `zk://host:port/path/state/`.
27It will not backup the state of the leader election: `zk://host:port/path/leader/`
28
29A backup is a single file in [tar](http://www.gnu.org/software/tar/) format, which contains all elements as file entries.
30
31### Options to create a backup
32
33There are multiple ways to create a backup:
34
35- cmd line flag `--backup_location {URI}`
36Marathon can be started with the command line option: `--backup_location` to a configurable URI.
37This option is strongly recommended because it will automatically create a backup of Marathon's state each time Marathon performs a migration.
38A migration usually happens during an upgrade of Marathon.
39
jeschkiesAuthorUnsubmitted

I wonder if the comma is needed in state whenever.

Is the backup of the state before or after the migration?

40- API endpoint `DELETE /v2/leader?backup={URI}` (experimental)
41You can create a backup at any time using the leader endpoint.
42The API call above will force the current leader to abdicate. The next leader will create a backup before it takes over leadership.
43This method is experimental: the API endpoint may change in the future.
44
45- Executable packaged with Marathon
jeschkiesAuthorUnsubmitted

Maybe The newly elected leader will ....

46The default Marathon package ships with a start script you can use to make a backup. </br>
47**Note**: This command connects directly to ZooKeeper and backs up state from the location you defined.
48Only use this method if there is no Marathon instance running.
49
50
51```
52$> marathon -main mesosphere.marathon.core.storage.backup.Backup --help
53$> marathon -main mesosphere.marathon.core.storage.backup.Backup --backup_location {backup_uri} --zk {zk_uri}
54```
55
56
57## Restore
58
59You can restore Marathon to a backup you have created.
60
61Every restore operation will perform the following steps:
62- Remove the current state in ZooKeeper under the defined path.
63- Restore the state from the backup to the defined ZooKeeper path.
64- The next Marathon leader migrates this state to the current version.
65
jeschkiesAuthorUnsubmitted

Hm, why the next Marathon leader?

aquamatthiasUnsubmitted

Is this question related to the documentation or logic?

jeschkiesAuthorUnsubmitted

Actually both:

  1. Do I have to delete the current leader for a backup?
  2. Would new Marathon leader be a better fit?
66### Options to restore from a backup
67
68There are multiple ways to restore a backup:
69
70- API endpoint `DELETE /v2/leader?restore={URI}` (experimental)
71You can use the `leader` endpoint to instruct a running Marathon instance to restore its state from backup.
72The API call above will force the current leader to abdicate.
73The next leader will perform a clean and restore operation, do a migration (if needed), and start up.
74This method is experimental: the API endpoint may change in the future.
75
76- Executable packaged with Marathon
77The default Marathon package ships with a start script you can use to restore from a backup.
78**Note:** This command connects directly to ZooKeepter and restores the state from the location you defined.
79Only use this method if there is no Marathon instance running.
80
81
82```
83$> marathon -main mesosphere.marathon.core.storage.backup.Restore --help
84$> marathon -main mesosphere.marathon.core.storage.backup.Restore --backup_location {backup_uri} --zk {zk_uri}
85```
86
87
88## Backup Storage Provider
89
90The backup & restore functionality allows different storage providers to save and load a backup.
91Currently, `file` and `s3` providers are implemented.
92If you need additional providers, please create a feature request in our [issue tracker](https://jira.mesosphere.com/secure/CreateIssue!default.jspa?pid=10401)
93
94### File storage
95This provider uses the file system to store and load a backup.
96It uses the default file URI syntax.
97
98Example: `file:///path/to/file`
99
100Attention: do not use the local file system if you are running a multi-master setup or the leading master can run on different machines.
101The backup and restore operation could be performed on different machines and the backup file might not be accessible.
102Use a shared file system to use the file storage provider in such a setup.
103
104### S3 storage
105This provider will use Amazon S3 to store and load a backup.
106Define an S3 location with the following URI syntax. See [S3 tools](http://s3tools.org/s3_about) for more information.
107
108Example: `s3://bucket-name/key/in/bucket`
109
110The following query parameters are supported:
111- `region` - the s3 region. Defaults to `us-east-1`
112- `access_key` and `secret_key` - to define the s3 credentials in the URI.
113 Only define these parameters in a fully secured environment: they will be stored in ZooKeeper and also visible in the logs.
114 See the provider chain to make credentials available without defining them in the URI.
115
116Provider chain:
117
118If credentials are not provided via the URI, the [AWS default credentials provider chain](http://docs.aws.amazon.com/sdk-for-java/v1/developer-guide/credentials.html) is used to look up aws credentials.