Open IT Experts for Enterprise

Zylk empresa de desarrollo de ecommerce

Snapshots in Elastic Search 7

Cesar Capillas
Cesar Capillas
Elastic logo

How to backup data in Elastic

Snapshots are a very important task for backup, restoration and upgrade procedures in Elastic Search. A snapshot is just a backup taken from a running ES cluster, what we often know as a hot backup. A big disclaimer is taken from documentation; you cannot back up an Elasticsearch cluster by simply taking a copy of the data directories of all of its nodes, while it is running. The only reliable way to back up a cluster is by using the snapshot and restore functionality.

Snapshots in Elastic Search 7

Snapshots are taken incrementally for a set of defined indices via Snapshot API or Kibana snapshot interface, where it is also possible to automate the snapshots. It is necessary to register a repository configuration path for saving the corresponding snapshots. The rootpath for snapshots’ repositories may be defined locally, for example with a dedicated backup disk or mount point in Elastic server. This config is set in your elasticsearch.yml config file (or in your $JAVA_OPTS via -D option) with:

path.repo: /opt/data/snapshots

There are also plugins that support remote repositories on S3, HDFS, Azure and Google Cloud Storage.

In the Kibana interface, it is quite simple to register a shared filesystem repository, where we can set some options as chunk size or compression of the snapshots.

Snapshots in Elastic Search 7

If you prefer the Kibana Dev Console or a REST Client (even curl if you are a command line hero), you may do something like:

# Creating a repo called test under path.repo
PUT /_snapshot/test
{
  "type": "fs",
  "settings": {
    "location": "/opt/data/snapshots/test"
  }
}

GET /_snapshot/test

# Creating a snapshot called snapshot.20191213
PUT /_snapshot/test/snapshot.20191213?wait_for_completion=true
{
  "indices": "filebeat-alfaudit-*,filebeat-alfresco-*",
  "ignore_unavailable": true,
  "include_global_state": false
}

# Get status
GET /_snapshot/test/snapshot.20191213/_status

# Delete your snapshot test
DELETE /_snapshot/test/snapshot.20191213

Finally, you can automate the generation of snapshots in Kibana UI: 

Snapshots in Elastic Search 7

Snapshots status may be checked via API or Elastic logs:

[2019-12-15T02:30:00,003][INFO ][o.e.x.s.SnapshotLifecycleTask] [pudu1] snapshot lifecycle policy [daily-snapshot] issuing create snapshot [elk741-2019.12.15-gjudab7hqgod6caapoazew]
[2019-12-15T02:30:00,005][INFO ][o.e.x.s.SnapshotLifecycleTask] [pudu1] snapshot lifecycle policy job [daily-snapshot-1] issued new snapshot creation for [elk741-2019.12.15-gjudab7hqgod6caapoazew] successfully
[2019-12-15T02:30:07,642][INFO ][o.e.s.SnapshotsService   ] [pudu1] snapshot [elk741:elk741-2019.12.15-gjudab7hqgod6caapoazew/q30iu1zITl68Zwm6D-PSYg] started
[2019-12-15T02:32:36,673][INFO ][o.e.s.SnapshotsService   ] [pudu1] snapshot [elk741:elk741-2019.12.15-gjudab7hqgod6caapoazew/q30iu1zITl68Zwm6D-PSYg] completed with state [SUCCESS]

Links:

Si te ha parecido interesante comparte este post en RRS

Facebook
LinkedIn
Telegram
Email

Leer más sobre temas relacionados

Deja un comentario

Tu dirección de correo electrónico no será publicada. Los campos obligatorios están marcados con *