Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 7 additions & 1 deletion .github/instructions/content.instructions.md
Original file line number Diff line number Diff line change
Expand Up @@ -88,7 +88,13 @@ Articles with `contentType: rai` in their frontmatter are **application or platf
* **Reusables**: RAI articles must only reference reusables from `data/reusables/rai/`. Place new RAI reusables there too.
* **Frontmatter**: New application cards use `contentType: rai`. The older `type: rai` is for legacy transparency notes not yet migrated.

## Parenthetical dashes
## Punctuation

### Semicolons

Avoid semicolons except where necessary. Generally they can be avoided and shorter sentences used instead. As a general rule, try to avoid very long sentences.

### Parenthetical dashes

Where a sentence of normal body text contains a parenthetical dash, the dash should always be an em dash without spaces at either side. This rule does not apply to text within code blocks.

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -116,6 +116,14 @@ Backup snapshots created by {% data variables.product.prodname_enterprise_backup

For more information on advanced usage, see the [{% data variables.product.prodname_enterprise_backup_utilities %} README](https://github.com/github/backup-utils#readme) in the {% data variables.product.prodname_enterprise_backup_utilities %} project documentation.

{% ifversion ghes > 3.19 %}

<!-- The linked article is versioned `ghes: '>=3.20'`, so this paragraph must stay gated to matching versions. Without the gate, the link is unresolvable in 3.19 and earlier and rendering fails. -->

By default, {% data variables.product.prodname_enterprise_backup_utilities %} backs up search index data by copying files directly from disk. You can optionally configure {% data variables.location.product_location %} to use Elasticsearch's native, incremental snapshot functionality with a customer-managed cloud storage provider instead. For more information, see [AUTOTITLE](/admin/backing-up-and-restoring-your-instance/configuring-elasticsearch-snapshots).

{% endif %}

## Upgrading {% data variables.product.prodname_enterprise_backup_utilities %}

When upgrading {% data variables.product.prodname_enterprise_backup_utilities %}, you must choose a version that will work with your current version of {% data variables.product.prodname_ghe_server %}. Your installation of {% data variables.product.prodname_enterprise_backup_utilities %} must be at least the same version as {% data variables.location.product_location %}, and cannot be more than two versions ahead. For more information, see [{% data variables.product.prodname_ghe_server %} version requirements](https://github.com/github/backup-utils/blob/master/docs/requirements.md#github-enterprise-server-version-requirements) in the {% data variables.product.prodname_enterprise_backup_utilities %} project documentation.
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,171 @@
---
title: Configuring Elasticsearch snapshots
shortTitle: Elasticsearch snapshots
intro: 'You can configure {% data variables.location.product_location %} to use a customer-managed cloud storage provider for native Elasticsearch snapshots and restores, instead of copying search index data directly from disk.'
permissions: Site administrators can configure Elasticsearch snapshot storage.
versions:
ghes: '>=3.20'
contentType: how-tos
category:
- Back up and upgrade your instance
---

<!-- TODO(docs review): confirm the minimum GHES version for this feature before merging. -->

## About Elasticsearch snapshots

By default, {% data variables.product.prodname_enterprise_backup_utilities %} and {% data variables.product.prodname_enterprise_backup_service %} back up search index data by copying files directly from disk. This approach doesn't follow Elasticsearch's own recommendations, can consume a large amount of backup appliance storage, and carries some risk of corrupting the backed-up indices.

As an opt-in alternative, {% data variables.location.product_location %} supports Elasticsearch's built-in, incremental snapshot and restore functionality. When you configure a supported cloud storage provider and enable the snapshot backup strategy:

* `ghe-backup` creates an incremental snapshot of your search index data in your configured cloud storage account. Incremental snapshots are faster than a full copy, especially on subsequent backups.
* `ghe-restore` restores search index data from your cloud storage account, if a snapshot exists that matches the target instance's {% data variables.product.prodname_ghe_server %} version.
* Snapshots are isolated per {% data variables.product.prodname_ghe_server %} patch version, which allows for safer rollbacks between versions.

This feature is opt-in. If you don't configure a snapshot repository, {% data variables.location.product_location %} continues to back up search index data using the existing disk-based method.

## Prerequisites

* A supported cloud storage provider: Azure Blob storage, Amazon S3, or Google Cloud Storage (or a service that's compatible with one of these APIs).
* The storage container or bucket you plan to use as the snapshot repository. **You must create this container or bucket yourself.** {% data variables.product.prodname_ghe_server %} does not create it for you, and repository registration will fail if the container or bucket doesn't already exist.

## Configuring a snapshot storage provider

Configure the following settings using `ghe-config` over SSH. For more information, see [AUTOTITLE](/admin/administering-your-instance/administering-your-instance-from-the-command-line/accessing-the-administrative-shell-ssh).

You'll apply the configuration to register the snapshot repository with Elasticsearch. The repository is named `search_index_snapshots-VERSION`, where `VERSION` is the current {% data variables.product.prodname_ghe_server %} release version.

### Azure Blob storage

1. Set the following secrets:

```shell
ghe-config secrets.elasticsearch.snapshot-provider azure
ghe-config secrets.elasticsearch.snapshot.azure.container YOUR-CONTAINER
ghe-config secrets.elasticsearch.snapshot.azure.account-name YOUR-STORAGE-ACCOUNT
ghe-config secrets.elasticsearch.snapshot.azure.account-key YOUR-ACCOUNT-KEY
```

Optionally, set a custom blob endpoint suffix (defaults to `core.windows.net`):

```shell
ghe-config secrets.elasticsearch.snapshot.azure.endpoint-suffix YOUR-ENDPOINT-SUFFIX
```

1. Run configuration apply.

```shell
ghe-config-apply
```

### Amazon S3

1. Set the following secrets:

```shell
ghe-config secrets.elasticsearch.snapshot-provider s3
ghe-config secrets.elasticsearch.snapshot.bucket-name YOUR-BUCKET-NAME
ghe-config secrets.elasticsearch.snapshot.service-url YOUR-SERVICE-URL
ghe-config secrets.elasticsearch.snapshot.region YOUR-AWS-REGION
ghe-config secrets.elasticsearch.snapshot.access-key-id YOUR-ACCESS-KEY-ID
ghe-config secrets.elasticsearch.snapshot.access-secret YOUR-ACCESS-SECRET
```

For the service URL, use your S3 endpoint, for example `https://s3.us-east-1.amazonaws.com`.

1. Run configuration apply.

```shell
ghe-config-apply
```

### Google Cloud Storage

1. Create a JSON service account key with access to your bucket, then encode it as base64.

```shell
base64 -w0 service-account.json
```

1. Set the following secrets:

```shell
ghe-config secrets.elasticsearch.snapshot-provider gcs
ghe-config secrets.elasticsearch.snapshot.gcs.bucket YOUR-BUCKET-NAME
ghe-config secrets.elasticsearch.snapshot.gcs.credentials YOUR-BASE64-ENCODED-CREDENTIALS
```

1. Run configuration apply.

```shell
ghe-config-apply
```

### Verifying registration

After configuration apply completes, confirm the snapshot repository was registered.

```shell
curl -k "http://127.0.0.1:9200/_snapshot/search_index_snapshots-$(ghe-version -v)/_status"
```

If a required secret is missing, configuration apply fails while registering the repository. Update the secrets and rerun `ghe-config-apply` to retry.

## Enabling snapshots during backup and restore

Configuring a storage provider registers the snapshot repository, but you must also enable the snapshot backup strategy so `ghe-backup` and `ghe-restore` use it automatically.

* **{% data variables.product.prodname_enterprise_backup_service %}:** Set the strategy using `ghe-config` on the appliance.

```shell
ghe-config backup.es-backup-strategy snapshot
```

* **{% data variables.product.prodname_enterprise_backup_utilities %}:** Set the strategy in your `backup.config` file on the backup host.

```shell
GHE_ES_BACKUP_STRATEGY=snapshot
```

The default value is `rsync`, which preserves the existing disk-based backup behavior. {% data variables.product.prodname_enterprise_backup_utilities %} stores only snapshot metadata locally; snapshot contents remain in your configured cloud storage.

If you don't set a backup strategy, the snapshot repository can still be managed manually, but `ghe-backup` and `ghe-restore` will continue to use the disk-based method.

## Managing snapshots manually

You can manage Elasticsearch snapshots directly using the following commands. Run any command with the `-h` flag for usage information.

| Command | Description |
| --- | --- |
| `ghe-es-create-snapshot` | Creates a new snapshot in the configured repository. |
| `ghe-es-list-snapshots` | Lists snapshot repositories, or snapshots within a repository. |
| `ghe-es-restore-snapshot` | Restores search indices from a snapshot. |

> [!TIP]
> For routine backups and restores, use {% data variables.product.prodname_enterprise_backup_utilities %} or {% data variables.product.prodname_enterprise_backup_service %} instead of these commands directly. The backup tooling coordinates Elasticsearch snapshots with the rest of your instance's data to ensure a consistent backup or restore. Use the manual commands only when you need to manage search index snapshots independently of a full instance backup or restore.

### Restoring a snapshot from an earlier version

You can restore a snapshot created by an earlier {% data variables.product.prodname_ghe_server %} version to a later version. You cannot restore a snapshot from a later version to an earlier version.

1. List available snapshots for the earlier version.

```shell
ghe-es-list-snapshots -v 3.14.0
```

1. Restore the snapshot you want.

```shell
ghe-es-restore-snapshot -v 3.14.0 -s SNAPSHOT-NAME
```

Elasticsearch automatically upgrades the restored indices to the current version's format.

## Managing snapshot storage

Elasticsearch snapshots are incremental, but your cloud storage account can still accumulate old snapshots over time. To avoid unbounded storage growth, configure a lifecycle policy with your storage provider to automatically delete snapshots older than your retention requirements. For more information, see your provider's documentation:

* [Amazon S3 Lifecycle rules](https://docs.aws.amazon.com/AmazonS3/latest/userguide/object-lifecycle-mgmt.html)
* [Azure Blob storage lifecycle management](https://learn.microsoft.com/en-us/azure/storage/blobs/lifecycle-management-overview)
* [Google Cloud Storage Object Lifecycle Management](https://cloud.google.com/storage/docs/lifecycle)
Original file line number Diff line number Diff line change
Expand Up @@ -179,3 +179,13 @@ Once the service is configured, you can define a backup schedule.
{% endif %}

The first run will be a full backup. Future runs will be incremental. If a new backup attempt starts while a previous one is still running, it may be skipped or fail. In that case, adjust the schedule to avoid overlap.

{% ifversion ghes > 3.19 %}

<!-- The linked article is versioned `ghes: '>=3.20'`, so this whole section must stay gated to matching versions. Without the gate, the link is unresolvable in 3.19 and earlier and rendering fails. -->

## Using Elasticsearch snapshots for search index data

By default, search index data is backed up by copying files directly from disk. You can optionally configure {% data variables.location.product_location %} to use Elasticsearch's native, incremental snapshot functionality with a customer-managed cloud storage provider instead. For more information, see [AUTOTITLE](/admin/backing-up-and-restoring-your-instance/configuring-elasticsearch-snapshots).
{% endif %}
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,7 @@ children:
- /about-the-backup-service-for-github-enterprise-server
- /understanding-the-backup-service
- /configuring-the-backup-service
- /configuring-elasticsearch-snapshots
- /creating-and-monitoring-backups
- /restoring-from-a-backup
- /configuring-remote-archives-for-backups
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,6 @@ redirect_from:
- /github/authenticating-to-github/authorizing-a-personal-access-token-for-use-with-saml-single-sign-on
- /github/authenticating-to-github/authenticating-with-saml-single-sign-on/authorizing-a-personal-access-token-for-use-with-saml-single-sign-on
- /authentication/authenticating-with-saml-single-sign-on/authorizing-a-personal-access-token-for-use-with-saml-single-sign-on
- /authentication/authenticating-with-single-sign-on/authorizing-credentials-for-single-sign-on-with-a-github-app
versions:
ghec: '*'
shortTitle: '{% data variables.product.pat_generic_caps %} with SSO'
Expand All @@ -20,6 +19,10 @@ You must authorize your {% data variables.product.pat_v1 %} after creation befor

{% data reusables.saml.authorized-creds-info %}

Enterprise administrators can use a {% data variables.product.prodname_github_app %} to authorize credentials for multiple organizations. See [AUTOTITLE](/authentication/authenticating-with-single-sign-on/authorizing-credentials-for-single-sign-on-with-a-github-app).

## Authorizing a {% data variables.product.pat_v1 %}

{% data reusables.user-settings.access_settings %}
{% data reusables.user-settings.developer_settings %}
{% data reusables.user-settings.personal_access_tokens %}
Expand Down
Loading
Loading