Feature/import export encryption (#1533)

Add option to encrypt repository exports with a password and add possibility to decrypt them on repository import. Also make the repository export asynchronous. This implies that the repository export will be created on the server and can be downloaded multiple times. The repository export will be deleted automatically 10 days after creation.
This commit is contained in:
Eduard Heimbuch
2021-02-25 13:01:03 +01:00
committed by GitHub
parent 367d7294b8
commit db2ce98721
53 changed files with 2698 additions and 237 deletions

Binary file not shown.

After

Width:  |  Height:  |  Size: 79 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 67 KiB

View File

@@ -48,6 +48,8 @@ Zusätzlich zum normalen Repository Import gibt es die Möglichkeit ein Reposito
Dieses Repository Archiv muss von einem anderen SCM-Manager exportiert worden sein und wird vor dem Import auf
Kompatibilität der Daten überprüft (der SCM-Manager und alle installierten Plugins müssen mindestens die Version des
exportierenden Systems haben).
Ist die zu importierende Datei verschlüsselt, muss das korrekte Passwort zum Entschlüsseln mitgeliefert werden.
Wird kein Passwort gesetzt, geht der SCM-Manager davon aus, dass die Datei unverschlüsselt ist.
![Repository importieren](assets/import-repository.png)

View File

@@ -18,7 +18,12 @@ Strategie `benutzerdefiniert` ausgewählt ist, kann zusätzlich zum Repository N
Ein archiviertes Repository kann nicht mehr verändert werden.
In dem Bereich "Repository exportieren" kann das Repository in unterschiedlichen Formaten exportiert werden.
Während des laufenden Exports kann auf das Repository nur lesend zugriffen werden.
Während eines laufenden Exports, kann auf das Repository nur lesend zugriffen werden.
Der Repository Export wird asynchron erstellt und auf dem Server gespeichert.
Existiert bereits ein Export für dieses Repository auf dem Server, wird dieser vorher gelöscht, da es immer nur einen Export pro Repository geben kann.
Exporte werden 10 Tage nach deren Erstellung automatisch vom SCM-Server gelöscht.
Falls ein Export existiert, wird über die blaue Info-Box angezeigt von wem, wann und wie dieser Export erzeugt wurde.
Das Ausgabeformat des Repository kann über die angebotenen Optionen verändert werden:
* `Standard`: Werden keine Optionen ausgewählt, wird das Repository im Standard Format exportiert.
Git und Mercurial werden dabei als `Tar Archiv` exportiert und Subversion nutzt das `Dump` Format.
@@ -27,8 +32,9 @@ Das Ausgabeformat des Repository kann über die angebotenen Optionen verändert
weitere Metadaten enthält. Für diesen Export sollte sichergestellt werden, dass alle installierten Plugins aktuell sind.
Ein Import eines so exportierten Repositories ist nur in einem SCM-Manager mit derselben oder einer neueren Version
möglich. Dieses gilt ebenso für alle installierten Plugins.
* `Verschlüsseln`: Die Export-Datei wird mit dem gesetzten Passwort verschlüsselt. Zum Entschlüsseln muss das exakt gleiche Passwort verwendet werden.
![Repository-Settings-General-Git](assets/repository-settings-general-git.png)
![Repository-Settings-General-Export](assets/repository-settings-general-export.png)
### Berechtigungen

Binary file not shown.

After

Width:  |  Height:  |  Size: 71 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 63 KiB

View File

@@ -46,6 +46,8 @@ In addition to the normal repository import, there is the possibility to import
This repository archive must have been exported from another SCM-Manager and is checked for data compatibility before
import (the SCM-Manager and all its installed plugins have to have at least the versions of the system the export has
been created on).
If the file to be imported is encrypted, the correct password must be supplied for decryption.
If no password is set, the SCM Manager assumes that the file is unencrypted.
![Import Repository](assets/import-repository.png)

View File

@@ -17,6 +17,11 @@ repository is marked as archived, it can no longer be modified.
In the "Export repository" section the repository can be exported in different formats.
During the export the repository cannot be modified!
When creating the export, the export file is saved on the server and can thus be downloaded repeatedly.
If an export already exists on the server, it will be deleted beforehand when a new export is created, as there can only ever be one export per repository.
Exports are automatically deleted from the SCM-Server 10 days after they are created.
If an export exists, the blue info box shows by whom, when and how this export was created.
The output format of the repository can be changed via the offered options:
* `Standard`: If no options are selected, the repository will be exported in the standard format.
Git and Mercurial are exported as `Tar archive` and Subversion uses the `Dump` format.
@@ -25,8 +30,9 @@ The output format of the repository can be changed via the offered options:
besides the repository. When you use this, please make sure all installed plugins are up to date. An import of
such an export is possible only in an SCM-Manager with the same or a newer version. The same is valid for all
installed plugins.
* `Encrypt`: The export file will be encrypted using the provided password. The same password must be used to decrypt this export file.
![Repository-Settings-General-Git](assets/repository-settings-general-git.png)
![Repository-Settings-General-Export](assets/repository-settings-general-export.png)
### Permissions

View File

@@ -0,0 +1,4 @@
- type: added
description: Add option to encrypt repository exports with a password and decrypt them on repository import ([#1533](https://github.com/scm-manager/scm-manager/pull/1533))
- type: added
description: Make repository export asynchronous. ([#1533](https://github.com/scm-manager/scm-manager/pull/1533))

View File

@@ -89,6 +89,9 @@ public class VndMediaType {
public static final String API_KEY = PREFIX + "apiKey" + SUFFIX;
public static final String API_KEY_COLLECTION = PREFIX + "apiKeyCollection" + SUFFIX;
public static final String REPOSITORY_EXPORT = PREFIX + "repositoryExport" + SUFFIX;
public static final String REPOSITORY_EXPORT_INFO = PREFIX + "repositoryExportInfo" + SUFFIX;
private VndMediaType() {
}

View File

@@ -24,8 +24,6 @@
package sonia.scm.repository;
import org.junit.jupiter.api.AfterEach;
import org.junit.jupiter.api.BeforeEach;
import org.junit.jupiter.api.Test;
import static org.assertj.core.api.AssertionsForClassTypes.assertThat;

View File

@@ -33,6 +33,8 @@ import static java.util.Optional.of;
class ExportableBlobFileStore extends ExportableDirectoryBasedFileStore {
private static final String EXCLUDED_EXPORT_STORE = "repository-export";
static final Function<StoreType, Optional<Function<Path, ExportableStore>>> BLOB_FACTORY =
storeType -> storeType == StoreType.BLOB ? of(ExportableBlobFileStore::new) : empty();
@@ -46,6 +48,9 @@ class ExportableBlobFileStore extends ExportableDirectoryBasedFileStore {
}
boolean shouldIncludeFile(Path file) {
if (getDirectory().toString().endsWith(EXCLUDED_EXPORT_STORE)) {
return false;
}
return file.getFileName().toString().endsWith(".blob");
}
}

View File

@@ -73,4 +73,8 @@ abstract class ExportableDirectoryBasedFileStore implements ExportableStore {
putFileContentIntoStream(exporter, fileOrDir);
}
}
protected Path getDirectory() {
return directory;
}
}

View File

@@ -0,0 +1,70 @@
/*
* MIT License
*
* Copyright (c) 2020-present Cloudogu GmbH and Contributors
*
* Permission is hereby granted, free of charge, to any person obtaining a copy
* of this software and associated documentation files (the "Software"), to deal
* in the Software without restriction, including without limitation the rights
* to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
* copies of the Software, and to permit persons to whom the Software is
* furnished to do so, subject to the following conditions:
*
* The above copyright notice and this permission notice shall be included in all
* copies or substantial portions of the Software.
*
* THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
* IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
* FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
* AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
* LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
* OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
* SOFTWARE.
*/
package sonia.scm.store;
import org.junit.jupiter.api.Test;
import java.nio.file.Path;
import java.nio.file.Paths;
import static org.assertj.core.api.Assertions.assertThat;
class ExportableBlobFileStoreTest {
@Test
void shouldIgnoreStoreIfExcludedStore() {
Path dir = Paths.get("test/path/repository-export");
ExportableBlobFileStore exportableBlobFileStore = new ExportableBlobFileStore(dir);
Path file = Paths.get(dir.toString(), "some.blob");
boolean result = exportableBlobFileStore.shouldIncludeFile(file);
assertThat(result).isFalse();
}
@Test
void shouldIgnoreStoreIfNotBlob() {
Path dir = Paths.get("test/path/any-store");
ExportableBlobFileStore exportableBlobFileStore = new ExportableBlobFileStore(dir);
Path file = Paths.get(dir.toString(), "some.unblob");
boolean result = exportableBlobFileStore.shouldIncludeFile(file);
assertThat(result).isFalse();
}
@Test
void shouldIncludeStore() {
Path dir = Paths.get("test/path/any-blob-store");
ExportableBlobFileStore exportableBlobFileStore = new ExportableBlobFileStore(dir);
Path file = Paths.get(dir.toString(), "some.blob");
boolean result = exportableBlobFileStore.shouldIncludeFile(file);
assertThat(result).isTrue();
}
}

View File

@@ -0,0 +1,129 @@
/*
* MIT License
*
* Copyright (c) 2020-present Cloudogu GmbH and Contributors
*
* Permission is hereby granted, free of charge, to any person obtaining a copy
* of this software and associated documentation files (the "Software"), to deal
* in the Software without restriction, including without limitation the rights
* to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
* copies of the Software, and to permit persons to whom the Software is
* furnished to do so, subject to the following conditions:
*
* The above copyright notice and this permission notice shall be included in all
* copies or substantial portions of the Software.
*
* THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
* IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
* FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
* AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
* LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
* OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
* SOFTWARE.
*/
package sonia.scm.store;
import java.io.ByteArrayInputStream;
import java.io.ByteArrayOutputStream;
import java.io.FileNotFoundException;
import java.io.IOException;
import java.io.InputStream;
import java.io.OutputStream;
import java.util.ArrayList;
import java.util.List;
import java.util.UUID;
public class InMemoryBlobStore implements BlobStore {
private final List<Blob> blobs = new ArrayList<>();
@Override
public Blob create() {
InMemoryBlob blob = new InMemoryBlob(UUID.randomUUID().toString());
blobs.add(blob);
return blob;
}
@Override
public Blob create(String id) {
InMemoryBlob blob = new InMemoryBlob(id);
blobs.add(blob);
return blob;
}
@Override
public void remove(Blob blob) {
blobs.remove(blob);
}
@Override
public List<Blob> getAll() {
return blobs;
}
@Override
public void clear() {
blobs.clear();
}
@Override
public void remove(String id) {
blobs.stream()
.filter(b -> b.getId().equals(id))
.findFirst()
.ifPresent(blobs::remove);
}
@Override
public Blob get(String id) {
return blobs.stream()
.filter(b -> b.getId().equals(id))
.findFirst()
.orElse(null);
}
private static class InMemoryBlob implements Blob {
private final String id;
private byte[] bytes = new byte[0];
private InMemoryBlob(String id) {
this.id = id;
}
@Override
public void commit() {
//Do nothing
}
@Override
public String getId() {
return id;
}
@Override
public InputStream getInputStream() throws FileNotFoundException {
return new ByteArrayInputStream(bytes);
}
@Override
public OutputStream getOutputStream() throws IOException {
return new InMemoryBlobByteArrayOutputStream();
}
@Override
public long getSize() {
return bytes.length;
}
private class InMemoryBlobByteArrayOutputStream extends ByteArrayOutputStream {
@Override
public void close() throws IOException {
bytes = super.toByteArray();
super.close();
}
}
}
}

View File

@@ -0,0 +1,107 @@
/*
* MIT License
*
* Copyright (c) 2020-present Cloudogu GmbH and Contributors
*
* Permission is hereby granted, free of charge, to any person obtaining a copy
* of this software and associated documentation files (the "Software"), to deal
* in the Software without restriction, including without limitation the rights
* to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
* copies of the Software, and to permit persons to whom the Software is
* furnished to do so, subject to the following conditions:
*
* The above copyright notice and this permission notice shall be included in all
* copies or substantial portions of the Software.
*
* THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
* IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
* FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
* AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
* LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
* OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
* SOFTWARE.
*/
package store;
import org.junit.jupiter.api.Test;
import sonia.scm.store.Blob;
import sonia.scm.store.InMemoryBlobStore;
import java.io.IOException;
import java.io.OutputStream;
import static org.assertj.core.api.Assertions.assertThat;
class InMemoryBlobStoreTest {
@Test
void shouldStoreToBlob() throws IOException {
String content = "SCM-Manager";
InMemoryBlobStore inMemoryBlobStore = new InMemoryBlobStore();
Blob blob = inMemoryBlobStore.create();
OutputStream os = blob.getOutputStream();
os.write(content.getBytes());
os.flush();
os.close();
byte[] result = new byte[11];
blob.getInputStream().read(result);
assertThat(new String(result)).isEqualTo(content);
}
@Test
void shouldGetBlobById() {
InMemoryBlobStore inMemoryBlobStore = new InMemoryBlobStore();
Blob first = inMemoryBlobStore.create("1");
inMemoryBlobStore.create("2");
inMemoryBlobStore.create("3");
assertThat(inMemoryBlobStore.get("1")).isEqualTo(first);
}
@Test
void shouldGetAllBlobs() {
InMemoryBlobStore inMemoryBlobStore = new InMemoryBlobStore();
Blob first = inMemoryBlobStore.create("1");
Blob second = inMemoryBlobStore.create("2");
Blob third = inMemoryBlobStore.create("3");
assertThat(inMemoryBlobStore.getAll()).contains(first, second, third);
}
@Test
void shouldRemoveBlobById() {
InMemoryBlobStore inMemoryBlobStore = new InMemoryBlobStore();
Blob blob = inMemoryBlobStore.create("1");
assertThat(inMemoryBlobStore.get("1")).isEqualTo(blob);
inMemoryBlobStore.remove("1");
assertThat(inMemoryBlobStore.getAll()).isEmpty();
}
@Test
void shouldRemoveBlob() {
InMemoryBlobStore inMemoryBlobStore = new InMemoryBlobStore();
Blob blob = inMemoryBlobStore.create("1");
assertThat(inMemoryBlobStore.get("1")).isEqualTo(blob);
inMemoryBlobStore.remove(blob);
assertThat(inMemoryBlobStore.getAll()).isEmpty();
}
@Test
void shouldClearBlobs() {
InMemoryBlobStore inMemoryBlobStore = new InMemoryBlobStore();
inMemoryBlobStore.create();
inMemoryBlobStore.create();
assertThat(inMemoryBlobStore.getAll()).hasSize(2);
inMemoryBlobStore.clear();
assertThat(inMemoryBlobStore.getAll()).isEmpty();
}
}

View File

@@ -23,12 +23,13 @@
*/
import {
ExportInfo,
Link,
Namespace,
Repository,
RepositoryCollection,
RepositoryCreation,
RepositoryTypeCollection,
RepositoryTypeCollection
} from "@scm-manager/ui-types";
import { useMutation, useQuery, useQueryClient } from "react-query";
import { apiClient } from "./apiclient";
@@ -37,6 +38,8 @@ import { createQueryString } from "./utils";
import { requiredLink } from "./links";
import { repoQueryKey } from "./keys";
import { concat } from "./urls";
import { useEffect, useState } from "react";
import { NotFoundError } from "./errors";
export type UseRepositoriesRequest = {
namespace?: Namespace;
@@ -227,3 +230,89 @@ export const useUnarchiveRepository = () => {
isUnarchived: !!data
};
};
export const useExportInfo = (repository: Repository): ApiResult<ExportInfo> => {
const link = requiredLink(repository, "exportInfo");
//TODO Refetch while exporting to update the page
const { isLoading, error, data } = useQuery<ExportInfo, Error>(
["repository", repository.namespace, repository.name, "exportInfo"],
() => apiClient.get(link).then(response => response.json()),
{}
);
return {
isLoading,
error: error instanceof NotFoundError ? null : error,
data
};
};
type ExportOptions = {
compressed: boolean;
withMetadata: boolean;
password?: string;
};
type ExportRepositoryMutateOptions = {
repository: Repository;
options: ExportOptions;
};
const EXPORT_MEDIA_TYPE = "application/vnd.scmm-repositoryExport+json;v=2";
export const useExportRepository = () => {
const queryClient = useQueryClient();
const [intervalId, setIntervalId] = useState<number | undefined>();
useEffect(() => {
return () => {
if (intervalId) {
clearInterval(intervalId);
}
};
}, [intervalId]);
const { mutate, isLoading, error, data } = useMutation<ExportInfo, Error, ExportRepositoryMutateOptions>(
({ repository, options }) => {
const infolink = requiredLink(repository, "exportInfo");
let link = requiredLink(repository, options.withMetadata ? "fullExport" : "export");
if (options.compressed) {
link += "?compressed=true";
}
return apiClient
.post(link, { password: options.password, async: true }, EXPORT_MEDIA_TYPE)
.then(() => queryClient.invalidateQueries(repoQueryKey(repository)))
.then(() => queryClient.invalidateQueries(["repositories"]))
.then(() => {
return new Promise<ExportInfo>((resolve, reject) => {
const id = setInterval(() => {
apiClient
.get(infolink)
.then(r => r.json())
.then((info: ExportInfo) => {
if (info._links.download) {
clearInterval(id);
resolve(info);
}
})
.catch(e => {
clearInterval(id);
reject(e);
});
}, 1000);
setIntervalId(id);
});
});
},
{
onSuccess: async (_, { repository }) => {
await queryClient.invalidateQueries(repoQueryKey(repository));
await queryClient.invalidateQueries(["repositories"]);
}
}
);
return {
exportRepository: (repository: Repository, options: ExportOptions) => mutate({ repository, options }),
isLoading,
error,
data
};
};

View File

@@ -33,7 +33,7 @@ export type RepositoryBase = NamespaceAndName & {
type: string;
contact?: string;
description?: string;
}
};
export type Repository = HalRepresentation &
RepositoryBase & {
@@ -53,6 +53,15 @@ export type RepositoryUrlImport = Repository & {
password?: string;
};
export type ExportInfo = HalRepresentation & {
exporterName: string;
created: Date;
withMetadata: boolean;
compressed: boolean;
encrypted: boolean;
status: "FINISHED" | "INTERRUPTED" | "EXPORTING";
};
export type Namespace = {
namespace: string;
_links: Links;

View File

@@ -74,11 +74,19 @@
},
"bundle": {
"title": "Wählen Sie Ihre Datei aus",
"helpText": "Wählen Sie die Datei aus der das Repository importiert werden soll."
"helpText": "Wählen Sie die Datei aus der das Repository importiert werden soll.",
"password": {
"title": "Passwort",
"helpText": "Wenn der importierte Dump verschlüsselt ist, muss das korrekte Passwort mitgeliefert werden."
}
},
"fullImport": {
"title": "SCM-Manager Repository Archiv",
"helpText": "Wählen Sie das Repository Archiv aus. Das Archiv muss von einer SCM-Manager Instanz exportiert worden sein. Diese Instanz und alle installierten Plugins müssen dieselbe oder eine neuere Version haben."
"helpText": "Wählen Sie das Repository Archiv aus. Das Archiv muss von einer SCM-Manager Instanz exportiert worden sein.",
"password": {
"title": "Passwort",
"helpText": "Wenn das importierte Repository Archiv verschlüsselt ist, muss das korrekte Passwort mitgeliefert werden."
}
},
"pending": {
"subtitle": "Repository wird importiert...",
@@ -254,7 +262,7 @@
},
"export": {
"subtitle": "Repository exportieren",
"notification": "Achtung: Während eines laufenden Exports kann auf das Repository nur lesend zugegriffen werden.",
"notification": "Achtung: Während eines laufenden Exports kann auf das Repository nur lesend zugegriffen werden. Erzeugte Repository Exporte werden nach 10 Tagen automatisch gelöscht.",
"compressed": {
"label": "Komprimieren",
"helpText": "Export Datei vor dem Download komprimieren. Reduziert die Downloadgröße."
@@ -263,8 +271,25 @@
"label": "Mit Metadaten (Experimentell)",
"helpText": "Zusätzlich zum Repository Dump werden Metadaten zum Repository und zur SCM-Instanz exportiert. Installierte Plugins sollten nach Möglichkeit in der neuesten Version installiert sein. Gespeicherte Passwörter funktionieren nicht bei einem Import in andere SCM-Manager Instanzen. Dieses Feature ist noch experimentell. Es sollte (noch) nicht für Backups genutzt werden!"
},
"exportButton": "Repository exportieren",
"exportStarted": "Der Repository Export wurde gestartet. Abhängig von der Größe des Repository kann dies einige Momente dauern."
"encrypt": {
"label": "Verschlüsseln",
"helpText": "Die Export Datei wird mit dem gesetzen Passwort verschlüsselt."
},
"password": {
"label": "Passwort",
"helpText": "Wird ein Passwort angegeben, wird die Export Datei damit verschlüsselt."
},
"createExportButton": "Neuen Export erstellen",
"downloadExportButton": "Export herunterladen",
"exportInfo": {
"infoBoxTitle": "Informationen zum gespeicherten Repository Export",
"exporter": "Erstellt von: {{username}}",
"created": "Erstellt am: ",
"repository": "Der Export enthält: \n- das Repository",
"repositoryArchive": "Der Export enthält: \n- das Repository\n- die Metadaten zum Repository, z. B. aus den Plugins\n- eine Umgebungsbeschreibung des exportierenden SCM-Managers\n- weitere Informationen zum Repository",
"encrypted": "Verschlüsselt: Der gespeicherte Export wurde verschlüsselt.",
"interrupted": "Unterbrochen: Der Export wurde während der Erstellung abgebrochen. Womöglich durch einen Server-Neustart."
}
},
"sources": {
"fileTree": {

View File

@@ -74,11 +74,19 @@
},
"bundle": {
"title": "Dump File",
"helpText": "Select your dump file from which the repository should be imported."
"helpText": "Select your dump file from which the repository should be imported.",
"password": {
"title": "Password",
"helpText": "If the imported dump have to be decrypted the correct password must be provided."
}
},
"fullImport": {
"title": "SCM-Manager Repository Archive",
"helpText": "Select the archive file which should be imported. The archive must have been exported from an SCM-Manager instance. This instance and all installed plugins have to be of the same or a newer version."
"helpText": "Select the archive file which should be imported. The archive must have been exported from an SCM-Manager instance.",
"password": {
"title": "Password",
"helpText": "If the imported repository archive have to be decrypted the correct password must be provided."
}
},
"pending": {
"subtitle": "Importing Repository...",
@@ -254,7 +262,7 @@
},
"export": {
"subtitle": "Repository Export",
"notification": "Attention: During the export the repository cannot be modified.",
"notification": "Attention: During the export the repository cannot be modified. Generated repository exports are automatically deleted after 10 days.",
"compressed": {
"label": "Compress",
"helpText": "Compress the export dump size to reduce the download size."
@@ -263,8 +271,25 @@
"label": "With metadata (Experimental)",
"helpText": "In addition to the repository dump, metadata about the repository and SCM instance is exported. If possible, ensure that installed plugins are up to date. However, stored passwords will not work if this is imported in other instances of SCM-Manager. This feature is still experimental. Do not use this as a backup mechanism (yet)!"
},
"exportButton": "Export Repository",
"exportStarted": "The repository export was started. Depending on the repository size this may take a while."
"encrypt": {
"label": "Encrypt",
"helpText": "The export file will be encrypted using the provided password."
},
"password": {
"label": "Password",
"helpText": "If a password is set, it will be used the encrypt the export file."
},
"createExportButton": "Create new Export",
"downloadExportButton": "Download Export",
"exportInfo": {
"infoBoxTitle": "Stored Repository Export Information",
"exporter": "Created by: {{username}}",
"created": "Created on: ",
"repository": "This export contains: \n- the repository",
"repositoryArchive": "This export contains: \n- the repository\n- the metadata which exist for this repository, e.g. from plugins\n- an environment description for the exporting SCM-Manager\n- additional information for this repository",
"encrypted": "Encrypted: The stored export has been encrypted.",
"interrupted": "Interrupted: The export was aborted during creation. Possibly due to a server restart."
}
},
"sources": {
"fileTree": {

View File

@@ -23,7 +23,7 @@
*/
import React, { FC } from "react";
import { FileUpload, LabelWithHelpIcon, Checkbox } from "@scm-manager/ui-components";
import { FileUpload, LabelWithHelpIcon, Checkbox, InputField } from "@scm-manager/ui-components";
import { File } from "@scm-manager/ui-types";
import { useTranslation } from "react-i18next";
@@ -32,34 +32,57 @@ type Props = {
setValid: (valid: boolean) => void;
compressed: boolean;
setCompressed: (compressed: boolean) => void;
password: string;
setPassword: (password: string) => void;
disabled: boolean;
};
const ImportFromBundleForm: FC<Props> = ({ setFile, setValid, compressed, setCompressed, disabled }) => {
const ImportFromBundleForm: FC<Props> = ({
setFile,
setValid,
compressed,
setCompressed,
password,
setPassword,
disabled
}) => {
const [t] = useTranslation("repos");
return (
<div className="columns">
<div className="column is-half is-vcentered">
<LabelWithHelpIcon label={t("import.bundle.title")} helpText={t("import.bundle.helpText")} />
<FileUpload
handleFile={(file: File) => {
setFile(file);
setValid(!!file);
}}
/>
<>
<div className="columns">
<div className="column is-half is-vcentered">
<LabelWithHelpIcon label={t("import.bundle.title")} helpText={t("import.bundle.helpText")} />
<FileUpload
handleFile={(file: File) => {
setFile(file);
setValid(!!file);
}}
/>
</div>
<div className="column is-half is-vcentered">
<Checkbox
checked={compressed}
onChange={(value, name) => setCompressed(value)}
label={t("import.compressed.label")}
disabled={disabled}
helpText={t("import.compressed.helpText")}
title={t("import.compressed.label")}
/>
</div>
</div>
<div className="column is-half is-vcentered">
<Checkbox
checked={compressed}
onChange={(value, name) => setCompressed(value)}
label={t("import.compressed.label")}
disabled={disabled}
helpText={t("import.compressed.helpText")}
title={t("import.compressed.label")}
/>
<div className="columns">
<div className="column is-half is-vcentered">
<InputField
value={password}
onChange={value => setPassword(value)}
type="password"
label={t("import.bundle.password.title")}
helpText={t("import.bundle.password.helpText")}
/>
</div>
</div>
</div>
</>
);
};

View File

@@ -28,7 +28,6 @@ import RepositoryInformationForm from "./RepositoryInformationForm";
import { apiClient, ErrorNotification, Level, SubmitButton } from "@scm-manager/ui-components";
import { useTranslation } from "react-i18next";
import { useHistory } from "react-router-dom";
import ImportFromBundleForm from "./ImportFromBundleForm";
import ImportFullRepositoryForm from "./ImportFullRepositoryForm";
type Props = {
@@ -44,9 +43,9 @@ const ImportFullRepository: FC<Props> = ({ url, repositoryType, setImportPending
type: repositoryType,
contact: "",
description: "",
_links: {},
_links: {}
});
const [password, setPassword] = useState("");
const [valid, setValid] = useState({ namespaceAndName: false, contact: true, file: false });
const [loading, setLoading] = useState(false);
const [error, setError] = useState<Error | undefined>();
@@ -59,7 +58,7 @@ const ImportFullRepository: FC<Props> = ({ url, repositoryType, setImportPending
setLoading(loading);
};
const isValid = () => Object.values(valid).every((v) => v);
const isValid = () => Object.values(valid).every(v => v);
const submit = (event: FormEvent<HTMLFormElement>) => {
event.preventDefault();
@@ -69,7 +68,7 @@ const ImportFullRepository: FC<Props> = ({ url, repositoryType, setImportPending
apiClient
.postBinary(url, formData => {
formData.append("bundle", file, file?.name);
formData.append("repository", JSON.stringify(repo));
formData.append("repository", JSON.stringify({ ...repo, password }));
})
.then(response => {
const location = response.headers.get("Location");
@@ -91,7 +90,12 @@ const ImportFullRepository: FC<Props> = ({ url, repositoryType, setImportPending
return (
<form onSubmit={submit}>
<ErrorNotification error={error} />
<ImportFullRepositoryForm setFile={setFile} setValid={(file: boolean) => setValid({ ...valid, file })}/>
<ImportFullRepositoryForm
setFile={setFile}
password={password}
setPassword={setPassword}
setValid={(file: boolean) => setValid({ ...valid, file })}
/>
<hr />
<NamespaceAndNameFields
repository={repo}

View File

@@ -23,21 +23,23 @@
*/
import React, { FC } from "react";
import { FileUpload, LabelWithHelpIcon, Checkbox } from "@scm-manager/ui-components";
import { FileUpload, InputField, LabelWithHelpIcon } from "@scm-manager/ui-components";
import { File } from "@scm-manager/ui-types";
import { useTranslation } from "react-i18next";
type Props = {
setFile: (file: File) => void;
setValid: (valid: boolean) => void;
password: string;
setPassword: (password: string) => void;
};
const ImportFullRepositoryForm: FC<Props> = ({ setFile, setValid}) => {
const ImportFullRepositoryForm: FC<Props> = ({ setFile, setValid, password, setPassword }) => {
const [t] = useTranslation("repos");
return (
<div className="columns">
<div className="column is-vcentered">
<div className="column is-half is-vcentered">
<LabelWithHelpIcon label={t("import.fullImport.title")} helpText={t("import.fullImport.helpText")} />
<FileUpload
handleFile={(file: File) => {
@@ -46,6 +48,15 @@ const ImportFullRepositoryForm: FC<Props> = ({ setFile, setValid}) => {
}}
/>
</div>
<div className="column is-half is-vcentered">
<InputField
value={password}
onChange={value => setPassword(value)}
type="password"
label={t("import.bundle.password.title")}
helpText={t("import.bundle.password.helpText")}
/>
</div>
</div>
);
};

View File

@@ -45,12 +45,12 @@ const ImportRepositoryFromBundle: FC<Props> = ({ url, repositoryType, setImportP
description: "",
_links: {}
});
const [password, setPassword] = useState("");
const [valid, setValid] = useState({ namespaceAndName: false, contact: true, file: false });
const [loading, setLoading] = useState(false);
const [error, setError] = useState<Error | undefined>();
const [file, setFile] = useState<File | null>(null);
const [compressed, setCompressed] = useState(false);
const [compressed, setCompressed] = useState(true);
const history = useHistory();
const [t] = useTranslation("repos");
@@ -69,7 +69,7 @@ const ImportRepositoryFromBundle: FC<Props> = ({ url, repositoryType, setImportP
apiClient
.postBinary(compressed ? url + "?compressed=true" : url, formData => {
formData.append("bundle", file, file?.name);
formData.append("repository", JSON.stringify(repo));
formData.append("repository", JSON.stringify({ ...repo, password }));
})
.then(response => {
const location = response.headers.get("Location");
@@ -96,6 +96,8 @@ const ImportRepositoryFromBundle: FC<Props> = ({ url, repositoryType, setImportP
setValid={(file: boolean) => setValid({ ...valid, file })}
compressed={compressed}
setCompressed={setCompressed}
password={password}
setPassword={setPassword}
disabled={loading}
/>
<hr />

View File

@@ -25,6 +25,7 @@ import React, { FC } from "react";
import { Link, RepositoryType } from "@scm-manager/ui-types";
import { LabelWithHelpIcon, Radio } from "@scm-manager/ui-components";
import { useTranslation } from "react-i18next";
import styled from "styled-components";
type Props = {
repositoryType: RepositoryType;
@@ -33,6 +34,12 @@ type Props = {
disabled?: boolean;
};
const RadioGroup = styled.div`
label {
margin-right: 2rem;
}
`;
const ImportTypeSelect: FC<Props> = ({ repositoryType, importType, setImportType, disabled }) => {
const [t] = useTranslation("repos");
@@ -45,18 +52,20 @@ const ImportTypeSelect: FC<Props> = ({ repositoryType, importType, setImportType
return (
<>
<LabelWithHelpIcon label={t("import.importTypes.label")} key="import.importTypes.label" />
{(repositoryType._links.import as Link[]).map((type, index) => (
<Radio
name={type.name}
checked={importType === type.name}
value={type.name}
label={t(`import.importTypes.${type.name}.label`)}
helpText={t(`import.importTypes.${type.name}.helpText`)}
onChange={changeImportType}
key={index}
disabled={disabled}
/>
))}
<RadioGroup>
{(repositoryType._links.import as Link[]).map((type, index) => (
<Radio
name={type.name}
checked={importType === type.name}
value={type.name}
label={t(`import.importTypes.${type.name}.label`)}
helpText={t(`import.importTypes.${type.name}.helpText`)}
onChange={changeImportType}
key={index}
disabled={disabled}
/>
))}
</RadioGroup>
</>
);
};

View File

@@ -25,10 +25,9 @@ import React, { FC } from "react";
import { Redirect, useRouteMatch } from "react-router-dom";
import RepositoryForm from "../components/form";
import { Repository } from "@scm-manager/ui-types";
import { ErrorNotification, Subtitle } from "@scm-manager/ui-components";
import { ErrorNotification, Subtitle, urls } from "@scm-manager/ui-components";
import { ExtensionPoint } from "@scm-manager/ui-extensions";
import RepositoryDangerZone from "./RepositoryDangerZone";
import { urls } from "@scm-manager/ui-components";
import { useTranslation } from "react-i18next";
import ExportRepository from "./ExportRepository";
import { useIndexLinks, useUpdateRepository } from "@scm-manager/ui-api";
@@ -58,7 +57,7 @@ const EditRepo: FC<Props> = ({ repository }) => {
<Subtitle subtitle={t("repositoryForm.subtitle")} />
<ErrorNotification error={error} />
<RepositoryForm repository={repository} loading={isLoading} modifyRepository={update} />
<ExportRepository repository={repository} />
{repository._links.exportInfo && <ExportRepository repository={repository} />}
<ExtensionPoint name="repo-config.route" props={extensionProps} renderAll={true} />
<RepositoryDangerZone repository={repository} indexLinks={indexLinks} />
</>

View File

@@ -21,70 +21,170 @@
* OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
* SOFTWARE.
*/
import React, { FC, useState } from "react";
import { Button, Checkbox, Level, Notification, Subtitle } from "@scm-manager/ui-components";
import React, { FC, useEffect, useState } from "react";
import {
Button,
ButtonGroup,
Checkbox,
DateShort,
ErrorNotification,
InputField,
Level,
Notification,
Subtitle
} from "@scm-manager/ui-components";
import { useTranslation } from "react-i18next";
import { Link, Repository } from "@scm-manager/ui-types";
import { ExportInfo, Link, Repository } from "@scm-manager/ui-types";
import { useExportInfo, useExportRepository } from "@scm-manager/ui-api";
import styled from "styled-components";
const InfoBox = styled.div`
white-space: pre-line;
background-color: #ccecf9;
margin: 1rem 0;
padding: 1rem;
border-radius: 2px;
border-left: 0.2rem solid;
border-color: #33b2e8;
`;
type Props = {
repository: Repository;
};
const ExportInterruptedNotification = () => {
const [t] = useTranslation("repos");
return <Notification type="warning">{t("export.exportInfo.interrupted")}</Notification>;
};
const ExportInfoBox: FC<{ exportInfo: ExportInfo }> = ({ exportInfo }) => {
const [t] = useTranslation("repos");
return (
<InfoBox>
<strong>{t("export.exportInfo.infoBoxTitle")}</strong>
<p>{t("export.exportInfo.exporter", { username: exportInfo.exporterName })}</p>
<p>
{t("export.exportInfo.created")}
<DateShort date={exportInfo.created} />
</p>
<br />
<p>{exportInfo.withMetadata ? t("export.exportInfo.repositoryArchive") : t("export.exportInfo.repository")}</p>
{exportInfo.encrypted && (
<>
<br />
<p>{t("export.exportInfo.encrypted")}</p>
</>
)}
</InfoBox>
);
};
const ExportRepository: FC<Props> = ({ repository }) => {
const [t] = useTranslation("repos");
const [compressed, setCompressed] = useState(true);
const [fullExport, setFullExport] = useState(false);
const [loading, setLoading] = useState(false);
const [encrypt, setEncrypt] = useState(false);
const [password, setPassword] = useState("");
const { isLoading: isLoadingInfo, error: errorInfo, data: exportInfo } = useExportInfo(repository);
const {
isLoading: isLoadingExport,
error: errorExport,
data: exportedInfo,
exportRepository
} = useExportRepository();
const createExportLink = () => {
if (fullExport) {
return (repository?._links?.fullExport as Link).href;
} else {
let exportLink = (repository?._links.export as Link).href;
if (compressed) {
exportLink += "?compressed=true";
}
return exportLink;
useEffect(() => {
if (exportedInfo && exportedInfo?._links.download) {
window.location.href = (exportedInfo?._links.download as Link).href;
}
};
}, [exportedInfo]);
if (!repository?._links?.export) {
if (!repository._links.export) {
return null;
}
const renderExportInfo = () => {
if (!exportInfo) {
return null;
}
if (exportInfo.status === "INTERRUPTED") {
return <ExportInterruptedNotification />;
} else {
return <ExportInfoBox exportInfo={exportInfo} />;
}
};
return (
<>
<hr />
<Subtitle subtitle={t("export.subtitle")} />
<Notification type="inherit">
{t("export.notification")}
</Notification>
<>
<ErrorNotification error={errorInfo} />
<ErrorNotification error={errorExport} />
<Notification type="inherit">{t("export.notification")}</Notification>
<Checkbox
checked={fullExport || compressed}
label={t("export.compressed.label")}
onChange={setCompressed}
helpText={t("export.compressed.helpText")}
disabled={fullExport}
/>
{repository?._links?.fullExport && (
<Checkbox
checked={fullExport || compressed}
label={t("export.compressed.label")}
onChange={setCompressed}
helpText={t("export.compressed.helpText")}
disabled={fullExport}
checked={fullExport}
label={t("export.fullExport.label")}
onChange={setFullExport}
helpText={t("export.fullExport.helpText")}
/>
{repository?._links?.fullExport && (
<Checkbox
checked={fullExport}
label={t("export.fullExport.label")}
onChange={setFullExport}
helpText={t("export.fullExport.helpText")}
)}
<Checkbox
checked={encrypt}
label={t("export.encrypt.label")}
onChange={setEncrypt}
helpText={t("export.encrypt.helpText")}
/>
{encrypt && (
<div className="columns column is-half">
<InputField
label={t("export.password.label")}
helpText={t("export.password.helpText")}
value={password}
onChange={setPassword}
type="password"
/>
)}
<Level
right={
<a color="primary" href={createExportLink()} onClick={() => setLoading(true)}>
<Button color="primary" label={t("export.exportButton")} icon="file-export" />
</div>
)}
{renderExportInfo()}
<Level
right={
<ButtonGroup>
<a color="info" href={(exportInfo?._links.download as Link)?.href}>
<Button
color="info"
disabled={isLoadingInfo || isLoadingExport || !exportInfo?._links.download}
label={t("export.downloadExportButton")}
icon="download"
/>
</a>
}
/>
{loading && <Notification onClose={() => setLoading(false)}>{t("export.exportStarted")}</Notification>}
</>
<Button
color="primary"
action={() =>
exportRepository(repository, {
compressed,
password: encrypt ? password : "",
withMetadata: fullExport
})
}
loading={isLoadingInfo || isLoadingExport}
disabled={isLoadingInfo || isLoadingExport}
label={t("export.createExportButton")}
icon="file-export"
/>
</ButtonGroup>
}
/>
</>
);
};
export default ExportRepository;

View File

@@ -37,12 +37,6 @@ import ImportFullRepository from "../components/ImportFullRepository";
import { useRepositoryTypes } from "@scm-manager/ui-api";
import { Prompt } from "react-router-dom";
type Props = {
repositoryTypes: RepositoryType[];
pageLoading: boolean;
error?: Error;
};
const ImportPendingLoading = ({ importPending }: { importPending: boolean }) => {
const [t] = useTranslation("repos");
if (!importPending) {

View File

@@ -88,5 +88,7 @@ public class MapperModule extends AbstractModule {
bind(PluginDtoMapper.class).to(Mappers.getMapperClass(PluginDtoMapper.class));
bind(ApiKeyToApiKeyDtoMapper.class).to(Mappers.getMapperClass(ApiKeyToApiKeyDtoMapper.class));
bind(RepositoryExportInformationToDtoMapper.class).to(Mappers.getMapperClass(RepositoryExportInformationToDtoMapper.class));
}
}

View File

@@ -0,0 +1,52 @@
/*
* MIT License
*
* Copyright (c) 2020-present Cloudogu GmbH and Contributors
*
* Permission is hereby granted, free of charge, to any person obtaining a copy
* of this software and associated documentation files (the "Software"), to deal
* in the Software without restriction, including without limitation the rights
* to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
* copies of the Software, and to permit persons to whom the Software is
* furnished to do so, subject to the following conditions:
*
* The above copyright notice and this permission notice shall be included in all
* copies or substantial portions of the Software.
*
* THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
* IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
* FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
* AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
* LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
* OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
* SOFTWARE.
*/
package sonia.scm.api.v2.resources;
import de.otto.edison.hal.HalRepresentation;
import de.otto.edison.hal.Links;
import lombok.Getter;
import lombok.NoArgsConstructor;
import lombok.Setter;
import sonia.scm.importexport.ExportStatus;
import java.time.Instant;
@Getter
@Setter
@NoArgsConstructor
@SuppressWarnings("squid:S2160") // we do not need equals for dto
public class RepositoryExportInformationDto extends HalRepresentation {
private String exporterName;
private Instant created;
private boolean withMetadata;
private boolean compressed;
private boolean encrypted;
private ExportStatus status;
RepositoryExportInformationDto(Links links) {
super(links);
}
}

View File

@@ -0,0 +1,70 @@
/*
* MIT License
*
* Copyright (c) 2020-present Cloudogu GmbH and Contributors
*
* Permission is hereby granted, free of charge, to any person obtaining a copy
* of this software and associated documentation files (the "Software"), to deal
* in the Software without restriction, including without limitation the rights
* to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
* copies of the Software, and to permit persons to whom the Software is
* furnished to do so, subject to the following conditions:
*
* The above copyright notice and this permission notice shall be included in all
* copies or substantial portions of the Software.
*
* THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
* IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
* FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
* AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
* LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
* OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
* SOFTWARE.
*/
package sonia.scm.api.v2.resources;
import com.google.common.annotations.VisibleForTesting;
import de.otto.edison.hal.Links;
import org.mapstruct.Context;
import org.mapstruct.Mapper;
import org.mapstruct.ObjectFactory;
import sonia.scm.importexport.ExportService;
import sonia.scm.importexport.ExportStatus;
import sonia.scm.importexport.RepositoryExportInformation;
import sonia.scm.repository.Repository;
import javax.inject.Inject;
import static de.otto.edison.hal.Link.link;
@Mapper
public abstract class RepositoryExportInformationToDtoMapper {
@Inject
private ResourceLinks resourceLinks;
@Inject
private ExportService exportService;
@VisibleForTesting
void setResourceLinks(ResourceLinks resourceLinks) {
this.resourceLinks = resourceLinks;
}
@VisibleForTesting
void setExportService(ExportService exportService) {
this.exportService = exportService;
}
abstract RepositoryExportInformationDto map(RepositoryExportInformation info, @Context Repository repository);
@ObjectFactory
RepositoryExportInformationDto createDto(@Context Repository repository) {
Links.Builder links = Links.linkingTo();
links.self(resourceLinks.repository().exportInfo(repository.getNamespace(), repository.getName()));
if (!exportService.isExporting(repository) && exportService.getExportInformation(repository).getStatus() == ExportStatus.FINISHED) {
links.single(link("download", resourceLinks.repository().downloadExport(repository.getNamespace(), repository.getName())));
}
return new RepositoryExportInformationDto(links.build());
}
}

View File

@@ -24,32 +24,47 @@
package sonia.scm.api.v2.resources;
import com.google.common.base.Strings;
import com.google.common.util.concurrent.ThreadFactoryBuilder;
import com.google.inject.Inject;
import io.swagger.v3.oas.annotations.Operation;
import io.swagger.v3.oas.annotations.media.Content;
import io.swagger.v3.oas.annotations.media.Schema;
import io.swagger.v3.oas.annotations.responses.ApiResponse;
import lombok.Getter;
import lombok.NoArgsConstructor;
import lombok.Setter;
import org.apache.commons.compress.compressors.gzip.GzipCompressorOutputStream;
import sonia.scm.BadRequestException;
import sonia.scm.ConcurrentModificationException;
import sonia.scm.NotFoundException;
import sonia.scm.Type;
import sonia.scm.importexport.ExportFileExtensionResolver;
import sonia.scm.importexport.ExportService;
import sonia.scm.importexport.FullScmRepositoryExporter;
import sonia.scm.repository.InternalRepositoryException;
import sonia.scm.importexport.RepositoryImportExportEncryption;
import sonia.scm.repository.NamespaceAndName;
import sonia.scm.repository.Repository;
import sonia.scm.repository.RepositoryManager;
import sonia.scm.repository.RepositoryPermissions;
import sonia.scm.repository.api.BundleCommandBuilder;
import sonia.scm.repository.api.Command;
import sonia.scm.repository.api.ExportFailedException;
import sonia.scm.repository.api.RepositoryService;
import sonia.scm.repository.api.RepositoryServiceFactory;
import sonia.scm.util.IOUtil;
import sonia.scm.web.VndMediaType;
import javax.validation.Valid;
import javax.validation.constraints.Pattern;
import javax.ws.rs.Consumes;
import javax.ws.rs.DELETE;
import javax.ws.rs.DefaultValue;
import javax.ws.rs.GET;
import javax.ws.rs.POST;
import javax.ws.rs.Path;
import javax.ws.rs.PathParam;
import javax.ws.rs.Produces;
import javax.ws.rs.QueryParam;
import javax.ws.rs.core.Context;
import javax.ws.rs.core.MediaType;
@@ -57,7 +72,12 @@ import javax.ws.rs.core.Response;
import javax.ws.rs.core.StreamingOutput;
import javax.ws.rs.core.UriInfo;
import java.io.IOException;
import java.io.InputStream;
import java.io.OutputStream;
import java.time.Instant;
import java.util.concurrent.Callable;
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
import static sonia.scm.ContextEntry.ContextBuilder.entity;
import static sonia.scm.api.v2.resources.RepositoryTypeSupportChecker.checkSupport;
@@ -65,16 +85,35 @@ import static sonia.scm.api.v2.resources.RepositoryTypeSupportChecker.type;
public class RepositoryExportResource {
private static final String NO_PASSWORD = "";
private final RepositoryManager manager;
private final RepositoryServiceFactory serviceFactory;
private final FullScmRepositoryExporter fullScmRepositoryExporter;
private final RepositoryImportExportEncryption repositoryImportExportEncryption;
private final ExecutorService repositoryExportHandler;
private final ExportService exportService;
private final RepositoryExportInformationToDtoMapper informationToDtoMapper;
private final ExportFileExtensionResolver fileExtensionResolver;
private final ResourceLinks resourceLinks;
@Inject
public RepositoryExportResource(RepositoryManager manager,
RepositoryServiceFactory serviceFactory, FullScmRepositoryExporter fullScmRepositoryExporter) {
RepositoryServiceFactory serviceFactory,
FullScmRepositoryExporter fullScmRepositoryExporter,
RepositoryImportExportEncryption repositoryImportExportEncryption,
ExportService exportService,
RepositoryExportInformationToDtoMapper informationToDtoMapper,
ExportFileExtensionResolver fileExtensionResolver, ResourceLinks resourceLinks) {
this.manager = manager;
this.serviceFactory = serviceFactory;
this.fullScmRepositoryExporter = fullScmRepositoryExporter;
this.repositoryImportExportEncryption = repositoryImportExportEncryption;
this.exportService = exportService;
this.informationToDtoMapper = informationToDtoMapper;
this.fileExtensionResolver = fileExtensionResolver;
this.resourceLinks = resourceLinks;
this.repositoryExportHandler = this.createExportHandlerPool();
}
/**
@@ -90,7 +129,6 @@ public class RepositoryExportResource {
*/
@GET
@Path("{type: ^(?!full$)[^/]+$}")
@Consumes(VndMediaType.REPOSITORY)
@Operation(summary = "Exports the repository", description = "Exports the repository.", tags = "Repository")
@ApiResponse(
responseCode = "200",
@@ -119,7 +157,8 @@ public class RepositoryExportResource {
@DefaultValue("false") @QueryParam("compressed") boolean compressed
) {
Repository repository = getVerifiedRepository(namespace, name, type);
return exportRepository(repository, compressed);
RepositoryPermissions.export(repository).check();
return exportRepository(repository, NO_PASSWORD, compressed, false);
}
/**
@@ -134,7 +173,6 @@ public class RepositoryExportResource {
*/
@GET
@Path("full")
@Consumes(VndMediaType.REPOSITORY)
@Operation(summary = "Exports the repository", description = "Exports the repository with metadata and environment information.", tags = "Repository")
@ApiResponse(
responseCode = "200",
@@ -161,11 +199,235 @@ public class RepositoryExportResource {
@PathParam("name") String name
) {
Repository repository = getVerifiedRepository(namespace, name);
return exportFullRepository(repository);
RepositoryPermissions.export(repository).check();
return exportFullRepository(repository, NO_PASSWORD, false);
}
/**
* Exports an existing repository without additional metadata. The method can
* only be used, if the repository type supports the {@link Command#BUNDLE}.
*
* @param uriInfo uri info
* @param namespace namespace of the repository
* @param name name of the repository
* @param type type of the repository
* @param request request of repository export which contains the password
* @return response with readable stream of repository dump
* @since 2.14.0
*/
@POST
@Path("{type: ^(?!full$)[^/]+$}")
@Consumes(VndMediaType.REPOSITORY_EXPORT)
@Operation(summary = "Exports the repository", description = "Exports the repository.", tags = "Repository")
@ApiResponse(
responseCode = "200",
description = "Repository export was successful"
)
@ApiResponse(
responseCode = "401",
description = "not authenticated / invalid credentials"
)
@ApiResponse(
responseCode = "403",
description = "not authorized, the current user has no privileges to read the repository"
)
@ApiResponse(
responseCode = "409",
description = "Repository export already started."
)
@ApiResponse(
responseCode = "500",
description = "internal server error",
content = @Content(
mediaType = VndMediaType.ERROR_TYPE,
schema = @Schema(implementation = ErrorDto.class)
)
)
public Response exportRepositoryWithPassword(@Context UriInfo uriInfo,
@PathParam("namespace") String namespace,
@PathParam("name") String name,
@Pattern(regexp = "\\w{1,10}") @PathParam("type") String type,
@DefaultValue("false") @QueryParam("compressed") boolean compressed,
@Valid ExportDto request
) throws Exception {
Repository repository = getVerifiedRepository(namespace, name, type);
RepositoryPermissions.export(repository).check();
checkRepositoryIsAlreadyExporting(repository);
return exportAsync(repository, request.isAsync(), () -> {
Response response = exportRepository(repository, request.getPassword(), compressed, request.isAsync());
exportService.setExportFinished(repository);
return response;
});
}
/**
* Exports an existing repository with all additional metadata and environment information. The method can
* only be used, if the repository type supports the {@link Command#BUNDLE}.
*
* @param uriInfo uri info
* @param namespace namespace of the repository
* @param name name of the repository
* @param request request of repository export which contains the password
* @return response with readable stream of repository dump
* @since 2.14.0
*/
@POST
@Path("full")
@Consumes(VndMediaType.REPOSITORY_EXPORT)
@Operation(summary = "Exports the repository", description = "Exports the repository with metadata and environment information.", tags = "Repository")
@ApiResponse(
responseCode = "200",
description = "Repository export was successful"
)
@ApiResponse(
responseCode = "204",
description = "Repository export was started successfully"
)
@ApiResponse(
responseCode = "401",
description = "not authenticated / invalid credentials"
)
@ApiResponse(
responseCode = "403",
description = "not authorized, the current user has no privileges to read the repository"
)
@ApiResponse(
responseCode = "500",
description = "internal server error",
content = @Content(
mediaType = VndMediaType.ERROR_TYPE,
schema = @Schema(implementation = ErrorDto.class)
)
)
public Response exportFullRepositoryWithPassword(@Context UriInfo uriInfo,
@PathParam("namespace") String namespace,
@PathParam("name") String name,
@Valid ExportDto request
) throws Exception {
Repository repository = getVerifiedRepository(namespace, name);
RepositoryPermissions.export(repository).check();
checkRepositoryIsAlreadyExporting(repository);
return exportAsync(repository, request.isAsync(), () -> {
Response response = exportFullRepository(repository, request.getPassword(), request.isAsync());
exportService.setExportFinished(repository);
return response;
});
}
@DELETE
@Path("")
@Operation(summary = "Deletes repository export", description = "Deletes repository export if stored.", tags = "Repository")
@ApiResponse(
responseCode = "204",
description = "Repository export was deleted"
)
@ApiResponse(
responseCode = "500",
description = "internal server error",
content = @Content(
mediaType = VndMediaType.ERROR_TYPE,
schema = @Schema(implementation = ErrorDto.class)
)
)
public Response deleteExport(@Context UriInfo uriInfo,
@PathParam("namespace") String namespace,
@PathParam("name") String name) {
Repository repository = getVerifiedRepository(namespace, name);
RepositoryPermissions.export(repository).check();
exportService.clear(repository.getId());
return Response.noContent().build();
}
@GET
@Path("download")
@Operation(summary = "Download stored repository export", description = "Download the stored repository export.", tags = "Repository")
@ApiResponse(
responseCode = "200",
description = "Repository export was downloaded"
)
@ApiResponse(
responseCode = "404",
description = "Repository export not found"
)
@ApiResponse(
responseCode = "409",
description = "Repository export is not ready yet"
)
@ApiResponse(
responseCode = "500",
description = "internal server error",
content = @Content(
mediaType = VndMediaType.ERROR_TYPE,
schema = @Schema(implementation = ErrorDto.class)
)
)
public Response downloadExport(@Context UriInfo uriInfo,
@PathParam("namespace") String namespace,
@PathParam("name") String name) {
Repository repository = getVerifiedRepository(namespace, name);
RepositoryPermissions.export(repository).check();
checkRepositoryIsAlreadyExporting(repository);
exportService.checkExportIsAvailable(repository);
StreamingOutput output = os -> {
try (InputStream is = exportService.getData(repository)) {
IOUtil.copy(is, os);
}
};
String fileExtension = exportService.getFileExtension(repository);
return createResponse(repository, fileExtension, fileExtension.contains(".gz"), output);
}
@GET
@Produces(VndMediaType.REPOSITORY_EXPORT_INFO)
@Path("info")
@Operation(summary = "Returns stored repository export information", description = "Returns the stored repository export information.", tags = "Repository")
@ApiResponse(
responseCode = "200",
description = "Repository export information"
)
@ApiResponse(
responseCode = "404",
description = "Repository export information not found"
)
@ApiResponse(
responseCode = "500",
description = "internal server error",
content = @Content(
mediaType = VndMediaType.ERROR_TYPE,
schema = @Schema(implementation = ErrorDto.class)
)
)
public RepositoryExportInformationDto getExportInformation(@Context UriInfo uriInfo,
@PathParam("namespace") String namespace,
@PathParam("name") String name) {
Repository repository = getVerifiedRepository(namespace, name);
RepositoryPermissions.export(repository).check();
return informationToDtoMapper.map(exportService.getExportInformation(repository), repository);
}
private Response exportAsync(Repository repository, boolean async, Callable<Response> call) throws Exception {
if (async) {
repositoryExportHandler.submit(call);
return Response.status(202).header(
"SCM-Export-Download",
resourceLinks.repository().downloadExport(repository.getNamespace(), repository.getName())
).build();
} else {
return call.call();
}
}
private void checkRepositoryIsAlreadyExporting(Repository repository) {
if (exportService.isExporting(repository)) {
throw new ConcurrentModificationException(Repository.class, repository.getId());
}
}
private Repository getVerifiedRepository(String namespace, String name) {
Repository repository = manager.get(new NamespaceAndName(namespace, name));
if (repository == null) {
throw new NotFoundException(Repository.class, namespace + "/" + name);
}
RepositoryPermissions.read().check(repository);
return repository;
}
@@ -181,53 +443,65 @@ public class RepositoryExportResource {
return repository;
}
private Response exportFullRepository(Repository repository) {
StreamingOutput output = os -> fullScmRepositoryExporter.export(repository, os);
private Response exportFullRepository(Repository repository, String password, boolean async) {
boolean encrypted = !Strings.isNullOrEmpty(password);
String fileExtension = fileExtensionResolver.resolve(repository, true, true, encrypted);
if (async) {
OutputStream blobOutputStream = exportService.store(repository, true, true, encrypted);
fullScmRepositoryExporter.export(repository, blobOutputStream, password);
exportService.setExportFinished(repository);
return Response.status(204).build();
} else {
StreamingOutput output = os -> fullScmRepositoryExporter.export(repository, os, password);
return Response
.ok(output, "application/x-gzip")
.header("content-disposition", createContentDispositionHeaderValue(repository, "tar.gz"))
.build();
return Response
.ok(output, "application/x-gzip")
.header("content-disposition", createContentDispositionHeaderValue(repository, fileExtension))
.build();
}
}
private Response exportRepository(Repository repository, boolean compressed) {
StreamingOutput output;
String fileExtension;
private Response exportRepository(Repository repository, String password, boolean compressed, boolean async) {
boolean encrypted = !Strings.isNullOrEmpty(password);
try (final RepositoryService service = serviceFactory.create(repository)) {
BundleCommandBuilder bundleCommand = service.getBundleCommand();
fileExtension = resolveFileExtension(bundleCommand, compressed);
output = os -> {
try {
if (compressed) {
GzipCompressorOutputStream gzipCompressorOutputStream = new GzipCompressorOutputStream(os);
bundleCommand.bundle(gzipCompressorOutputStream);
gzipCompressorOutputStream.finish();
} else {
bundleCommand.bundle(os);
}
} catch (IOException e) {
throw new InternalRepositoryException(repository, "repository export failed", e);
}
};
String fileExtension = fileExtensionResolver.resolve(repository, false, compressed, encrypted);
if (async) {
OutputStream blobOutputStream = exportService.store(repository, false, compressed, !Strings.isNullOrEmpty(password));
OutputStream os = repositoryImportExportEncryption.optionallyEncrypt(blobOutputStream, password);
bundleRepository(os, compressed, bundleCommand);
return Response.status(204).build();
} else {
StreamingOutput output = os -> {
os = repositoryImportExportEncryption.optionallyEncrypt(os, password);
bundleRepository(os, compressed, bundleCommand);
};
return createResponse(repository, fileExtension, compressed, output);
}
} catch (IOException e) {
throw new ExportFailedException(entity(repository).build(), "repository export failed", e);
}
return createResponse(repository, fileExtension, compressed, output);
}
private Response createResponse(Repository repository, String fileExtension, boolean compressed, StreamingOutput output) {
private void bundleRepository(OutputStream os, boolean compressed, BundleCommandBuilder bundleCommand) throws IOException {
if (compressed) {
GzipCompressorOutputStream gzipCompressorOutputStream = new GzipCompressorOutputStream(os);
bundleCommand.bundle(gzipCompressorOutputStream);
gzipCompressorOutputStream.finish();
} else {
bundleCommand.bundle(os);
}
}
private Response createResponse(Repository repository, String fileExtension, boolean compressed, StreamingOutput
output) {
return Response
.ok(output, compressed ? "application/x-gzip" : MediaType.APPLICATION_OCTET_STREAM)
.header("content-disposition", createContentDispositionHeaderValue(repository, fileExtension))
.build();
}
private String resolveFileExtension(BundleCommandBuilder bundleCommand, boolean compressed) {
if (compressed) {
return bundleCommand.getFileExtension() + ".gz";
} else {
return bundleCommand.getFileExtension();
}
}
private String createContentDispositionHeaderValue(Repository repository, String fileExtension) {
String timestamp = createFormattedTimestamp();
@@ -244,6 +518,15 @@ public class RepositoryExportResource {
return Instant.now().toString().replace(":", "-").split("\\.")[0];
}
private ExecutorService createExportHandlerPool() {
return Executors.newCachedThreadPool(
new ThreadFactoryBuilder()
.setNameFormat("RepositoryExportHandler-%d")
.build()
);
}
private static class WrongTypeException extends BadRequestException {
private static final String CODE = "4hSNNTBiu1";
@@ -257,4 +540,12 @@ public class RepositoryExportResource {
return CODE;
}
}
@Getter
@Setter
@NoArgsConstructor
private static class ExportDto {
private String password;
private boolean async;
}
}

View File

@@ -32,8 +32,6 @@ import com.google.common.base.Strings;
import com.google.common.io.ByteSource;
import com.google.common.io.Files;
import com.google.inject.Inject;
import de.otto.edison.hal.Embedded;
import de.otto.edison.hal.Links;
import io.swagger.v3.oas.annotations.Operation;
import io.swagger.v3.oas.annotations.media.Content;
import io.swagger.v3.oas.annotations.media.Schema;
@@ -47,10 +45,12 @@ import org.jboss.resteasy.plugins.providers.multipart.MultipartFormDataInput;
import org.jboss.resteasy.plugins.providers.multipart.MultipartInputImpl;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import sonia.scm.ContextEntry;
import sonia.scm.HandlerEventType;
import sonia.scm.Type;
import sonia.scm.event.ScmEventBus;
import sonia.scm.importexport.FullScmRepositoryImporter;
import sonia.scm.importexport.RepositoryImportExportEncryption;
import sonia.scm.repository.InternalRepositoryException;
import sonia.scm.repository.Repository;
import sonia.scm.repository.RepositoryImportEvent;
@@ -58,16 +58,15 @@ import sonia.scm.repository.RepositoryManager;
import sonia.scm.repository.RepositoryPermission;
import sonia.scm.repository.RepositoryPermissions;
import sonia.scm.repository.api.Command;
import sonia.scm.repository.api.ImportFailedException;
import sonia.scm.repository.api.PullCommandBuilder;
import sonia.scm.repository.api.RepositoryService;
import sonia.scm.repository.api.RepositoryServiceFactory;
import sonia.scm.util.IOUtil;
import sonia.scm.util.ValidationUtil;
import sonia.scm.web.VndMediaType;
import sonia.scm.web.api.DtoValidator;
import javax.validation.Valid;
import javax.validation.constraints.Email;
import javax.validation.constraints.NotEmpty;
import javax.validation.constraints.Pattern;
import javax.ws.rs.Consumes;
@@ -104,6 +103,7 @@ public class RepositoryImportResource {
private final ResourceLinks resourceLinks;
private final ScmEventBus eventBus;
private final FullScmRepositoryImporter fullScmRepositoryImporter;
private final RepositoryImportExportEncryption repositoryImportExportEncryption;
@Inject
public RepositoryImportResource(RepositoryManager manager,
@@ -111,13 +111,15 @@ public class RepositoryImportResource {
RepositoryServiceFactory serviceFactory,
ResourceLinks resourceLinks,
ScmEventBus eventBus,
FullScmRepositoryImporter fullScmRepositoryImporter) {
FullScmRepositoryImporter fullScmRepositoryImporter,
RepositoryImportExportEncryption repositoryImportExportEncryption) {
this.manager = manager;
this.mapper = mapper;
this.serviceFactory = serviceFactory;
this.resourceLinks = resourceLinks;
this.eventBus = eventBus;
this.fullScmRepositoryImporter = fullScmRepositoryImporter;
this.repositoryImportExportEncryption = repositoryImportExportEncryption;
}
/**
@@ -142,7 +144,7 @@ public class RepositoryImportResource {
description = "Repository import was successful",
content = @Content(
mediaType = VndMediaType.REPOSITORY,
schema = @Schema(implementation = ImportRepositoryDto.class)
schema = @Schema(implementation = ImportRepositoryFromUrlDto.class)
)
)
@ApiResponse(
@@ -163,7 +165,7 @@ public class RepositoryImportResource {
)
public Response importFromUrl(@Context UriInfo uriInfo,
@Pattern(regexp = "\\w{1,10}") @PathParam("type") String type,
@Valid RepositoryImportDto request) {
@Valid RepositoryImportResource.RepositoryImportFromUrlDto request) {
RepositoryPermissions.create().check();
Type t = type(manager, type);
@@ -192,7 +194,7 @@ public class RepositoryImportResource {
}
@VisibleForTesting
Consumer<Repository> pullChangesFromRemoteUrl(RepositoryImportDto request) {
Consumer<Repository> pullChangesFromRemoteUrl(RepositoryImportFromUrlDto request) {
return repository -> {
try (RepositoryService service = serviceFactory.create(repository)) {
PullCommandBuilder pullCommand = service.getPullCommand();
@@ -309,8 +311,9 @@ public class RepositoryImportResource {
private Repository importFullRepositoryFromInput(MultipartFormDataInput input) {
Map<String, List<InputPart>> formParts = input.getFormDataMap();
InputStream inputStream = extractInputStream(formParts);
RepositoryDto repositoryDto = extractRepositoryDto(formParts);
return fullScmRepositoryImporter.importFromStream(mapper.map(repositoryDto), inputStream);
RepositoryImportFromFileDto repositoryDto = extractRepositoryDto(formParts);
return fullScmRepositoryImporter.importFromStream(mapper.map(repositoryDto), inputStream, repositoryDto.getPassword());
}
/**
@@ -324,7 +327,8 @@ public class RepositoryImportResource {
private Repository doImportFromBundle(String type, MultipartFormDataInput input, boolean compressed) {
Map<String, List<InputPart>> formParts = input.getFormDataMap();
InputStream inputStream = extractInputStream(formParts);
RepositoryDto repositoryDto = extractRepositoryDto(formParts);
RepositoryImportFromFileDto repositoryDto = extractRepositoryDto(formParts);
inputStream = decryptInputStream(inputStream, repositoryDto.getPassword());
Type t = type(manager, type);
checkSupport(t, Command.UNBUNDLE);
@@ -349,6 +353,14 @@ public class RepositoryImportResource {
return repository;
}
private InputStream decryptInputStream(InputStream inputStream, String password) {
try {
return repositoryImportExportEncryption.decrypt(inputStream, password);
} catch (IOException e) {
throw new ImportFailedException(ContextEntry.ContextBuilder.noContext(), "import failed", e);
}
}
@VisibleForTesting
Consumer<Repository> unbundleImport(InputStream inputStream, boolean compressed) {
return repository -> {
@@ -370,8 +382,8 @@ public class RepositoryImportResource {
};
}
private RepositoryDto extractRepositoryDto(Map<String, List<InputPart>> formParts) {
RepositoryDto repositoryDto = extractFromInputPart(formParts.get("repository"), RepositoryDto.class);
private RepositoryImportFromFileDto extractRepositoryDto(Map<String, List<InputPart>> formParts) {
RepositoryImportFromFileDto repositoryDto = extractFromInputPart(formParts.get("repository"), RepositoryImportFromFileDto.class);
checkNotNull(repositoryDto, "repository data is required");
DtoValidator.validate(repositoryDto);
return repositoryDto;
@@ -416,32 +428,22 @@ public class RepositoryImportResource {
@Setter
@NoArgsConstructor
@SuppressWarnings("java:S2160")
public static class RepositoryImportDto extends RepositoryDto implements ImportRepositoryDto {
public static class RepositoryImportFromUrlDto extends RepositoryDto implements ImportRepositoryFromUrlDto {
@NotEmpty
private String importUrl;
private String username;
private String password;
RepositoryImportDto(Links links, Embedded embedded) {
super(links, embedded);
}
}
interface ImportRepositoryDto {
String getNamespace();
@Pattern(regexp = ValidationUtil.REGEX_REPOSITORYNAME)
String getName();
@NotEmpty
String getType();
@Email
String getContact();
String getDescription();
@Getter
@Setter
@NoArgsConstructor
@SuppressWarnings("java:S2160")
public static class RepositoryImportFromFileDto extends RepositoryDto implements ImportRepositoryFromFileDto {
private String password;
}
interface ImportRepositoryFromUrlDto extends CreateRepositoryDto {
@NotEmpty
String getImportUrl();
@@ -449,4 +451,8 @@ public class RepositoryImportResource {
String getPassword();
}
interface ImportRepositoryFromFileDto extends CreateRepositoryDto {
String getPassword();
}
}

View File

@@ -115,6 +115,7 @@ public abstract class RepositoryToRepositoryDtoMapper extends BaseMapper<Reposit
if (repositoryService.isSupported(Command.BUNDLE) && RepositoryPermissions.export(repository).isPermitted()) {
linksBuilder.single(link("export", resourceLinks.repository().export(repository.getNamespace(), repository.getName(), repository.getType())));
linksBuilder.single(link("fullExport", resourceLinks.repository().fullExport(repository.getNamespace(), repository.getName(), repository.getType())));
linksBuilder.single(link("exportInfo", resourceLinks.repository().exportInfo(repository.getNamespace(), repository.getName())));
}
if (repositoryService.isSupported(Command.TAGS)) {

View File

@@ -392,6 +392,14 @@ class ResourceLinks {
String fullExport(String namespace, String name, String type) {
return repositoryExportLinkBuilder.method("getRepositoryResource").parameters(namespace, name).method("export").parameters().method("exportFullRepository").parameters(type).href();
}
String downloadExport(String namespace, String name) {
return repositoryExportLinkBuilder.method("getRepositoryResource").parameters(namespace, name).method("export").parameters().method("downloadExport").parameters().href();
}
String exportInfo(String namespace, String name) {
return repositoryExportLinkBuilder.method("getRepositoryResource").parameters(namespace, name).method("export").parameters().method("getExportInformation").parameters().href();
}
}
RepositoryCollectionLinks repositoryCollection() {

View File

@@ -0,0 +1,59 @@
/*
* MIT License
*
* Copyright (c) 2020-present Cloudogu GmbH and Contributors
*
* Permission is hereby granted, free of charge, to any person obtaining a copy
* of this software and associated documentation files (the "Software"), to deal
* in the Software without restriction, including without limitation the rights
* to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
* copies of the Software, and to permit persons to whom the Software is
* furnished to do so, subject to the following conditions:
*
* The above copyright notice and this permission notice shall be included in all
* copies or substantial portions of the Software.
*
* THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
* IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
* FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
* AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
* LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
* OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
* SOFTWARE.
*/
package sonia.scm.importexport;
import sonia.scm.repository.Repository;
import sonia.scm.repository.api.RepositoryService;
import sonia.scm.repository.api.RepositoryServiceFactory;
import javax.inject.Inject;
public class ExportFileExtensionResolver {
private final RepositoryServiceFactory serviceFactory;
@Inject
public ExportFileExtensionResolver(RepositoryServiceFactory serviceFactory) {
this.serviceFactory = serviceFactory;
}
public String resolve(Repository repository, boolean withMetadata, boolean compressed, boolean encrypted) {
StringBuilder builder = new StringBuilder();
if (withMetadata) {
builder.append("tar.gz");
} else {
try (RepositoryService service = serviceFactory.create(repository)) {
builder.append(service.getBundleCommand().getFileExtension());
}
if (compressed) {
builder.append(".gz");
}
}
if (encrypted) {
builder.append(".enc");
}
return builder.toString();
}
}

View File

@@ -0,0 +1,48 @@
/*
* MIT License
*
* Copyright (c) 2020-present Cloudogu GmbH and Contributors
*
* Permission is hereby granted, free of charge, to any person obtaining a copy
* of this software and associated documentation files (the "Software"), to deal
* in the Software without restriction, including without limitation the rights
* to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
* copies of the Software, and to permit persons to whom the Software is
* furnished to do so, subject to the following conditions:
*
* The above copyright notice and this permission notice shall be included in all
* copies or substantial portions of the Software.
*
* THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
* IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
* FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
* AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
* LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
* OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
* SOFTWARE.
*/
package sonia.scm.importexport;
import sonia.scm.Initable;
import sonia.scm.SCMContextProvider;
import sonia.scm.schedule.Scheduler;
import javax.inject.Inject;
public class ExportGarbageCollector implements Initable {
private final ExportService exportService;
private final Scheduler scheduler;
@Inject
public ExportGarbageCollector(ExportService exportService, Scheduler scheduler) {
this.exportService = exportService;
this.scheduler = scheduler;
}
@Override
public void init(SCMContextProvider context) {
scheduler.schedule("0 0 6 * * ?", exportService::cleanupOutdatedExports);
}
}

View File

@@ -0,0 +1,199 @@
/*
* MIT License
*
* Copyright (c) 2020-present Cloudogu GmbH and Contributors
*
* Permission is hereby granted, free of charge, to any person obtaining a copy
* of this software and associated documentation files (the "Software"), to deal
* in the Software without restriction, including without limitation the rights
* to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
* copies of the Software, and to permit persons to whom the Software is
* furnished to do so, subject to the following conditions:
*
* The above copyright notice and this permission notice shall be included in all
* copies or substantial portions of the Software.
*
* THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
* IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
* FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
* AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
* LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
* OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
* SOFTWARE.
*/
package sonia.scm.importexport;
import org.apache.shiro.SecurityUtils;
import sonia.scm.NotFoundException;
import sonia.scm.repository.Repository;
import sonia.scm.repository.RepositoryPermissions;
import sonia.scm.repository.api.ExportFailedException;
import sonia.scm.store.Blob;
import sonia.scm.store.BlobStore;
import sonia.scm.store.BlobStoreFactory;
import sonia.scm.store.DataStore;
import sonia.scm.store.DataStoreFactory;
import sonia.scm.user.User;
import javax.inject.Inject;
import java.io.IOException;
import java.io.InputStream;
import java.io.OutputStream;
import java.time.Instant;
import java.time.temporal.ChronoUnit;
import java.util.ArrayList;
import java.util.List;
import java.util.Map;
import java.util.stream.Collectors;
import static sonia.scm.ContextEntry.ContextBuilder.entity;
public class ExportService {
static final String STORE_NAME = "repository-export";
private final BlobStoreFactory blobStoreFactory;
private final DataStoreFactory dataStoreFactory;
private final ExportFileExtensionResolver fileExtensionResolver;
@Inject
public ExportService(BlobStoreFactory blobStoreFactory, DataStoreFactory dataStoreFactory, ExportFileExtensionResolver fileExtensionResolver) {
this.blobStoreFactory = blobStoreFactory;
this.dataStoreFactory = dataStoreFactory;
this.fileExtensionResolver = fileExtensionResolver;
}
public OutputStream store(Repository repository, boolean withMetadata, boolean compressed, boolean encrypted) {
RepositoryPermissions.export(repository).check();
storeExportInformation(repository.getId(), withMetadata, compressed, encrypted);
try {
return storeNewBlob(repository.getId()).getOutputStream();
} catch (IOException e) {
throw new ExportFailedException(
entity(repository).build(),
"Could not store repository export to blob file",
e
);
}
}
public RepositoryExportInformation getExportInformation(Repository repository) {
RepositoryPermissions.export(repository).check();
RepositoryExportInformation info = createDataStore().get(repository.getId());
if (info == null) {
throw new NotFoundException(RepositoryExportInformation.class, repository.getId());
}
return info;
}
public InputStream getData(Repository repository) throws IOException {
RepositoryPermissions.export(repository).check();
Blob blob = getBlob(repository.getId());
if (blob == null) {
throw new NotFoundException(Blob.class, repository.getId());
}
return blob.getInputStream();
}
public void checkExportIsAvailable(Repository repository) {
RepositoryPermissions.export(repository).check();
if (createDataStore().get(repository.getId()) == null) {
throw new NotFoundException(RepositoryExportInformation.class, repository.getId());
}
}
public String getFileExtension(Repository repository) {
RepositoryPermissions.export(repository).check();
RepositoryExportInformation exportInfo = getExportInformation(repository);
return fileExtensionResolver.resolve(repository, exportInfo.isWithMetadata(), exportInfo.isCompressed(), exportInfo.isEncrypted());
}
public void clear(String repositoryId) {
RepositoryPermissions.export(repositoryId).check();
createDataStore().remove(repositoryId);
createBlobStore(repositoryId).clear();
}
public void setExportFinished(Repository repository) {
RepositoryPermissions.export(repository).check();
DataStore<RepositoryExportInformation> dataStore = createDataStore();
RepositoryExportInformation info = dataStore.get(repository.getId());
info.setStatus(ExportStatus.FINISHED);
dataStore.put(repository.getId(), info);
}
public boolean isExporting(Repository repository) {
RepositoryPermissions.export(repository).check();
RepositoryExportInformation info = createDataStore().get(repository.getId());
return info != null && info.getStatus() == ExportStatus.EXPORTING;
}
public void cleanupUnfinishedExports() {
DataStore<RepositoryExportInformation> dataStore = createDataStore();
List<Map.Entry<String, RepositoryExportInformation>> unfinishedExports = dataStore.getAll().entrySet().stream()
.filter(e -> e.getValue().getStatus() == ExportStatus.EXPORTING)
.collect(Collectors.toList());
for (Map.Entry<String, RepositoryExportInformation> export : unfinishedExports) {
createBlobStore(export.getKey()).clear();
RepositoryExportInformation info = dataStore.get(export.getKey());
info.setStatus(ExportStatus.INTERRUPTED);
dataStore.put(export.getKey(), info);
}
}
void cleanupOutdatedExports() {
DataStore<RepositoryExportInformation> dataStore = createDataStore();
List<String> outdatedExportIds = collectOutdatedExportIds(dataStore);
for (String id : outdatedExportIds) {
createBlobStore(id).clear();
}
outdatedExportIds.forEach(dataStore::remove);
}
private List<String> collectOutdatedExportIds(DataStore<RepositoryExportInformation> dataStore) {
List<String> outdatedExportIds = new ArrayList<>();
Instant expireDate = Instant.now().minus(10, ChronoUnit.DAYS);
dataStore
.getAll()
.entrySet()
.stream()
.filter(e -> e.getValue().getCreated().isBefore(expireDate))
.forEach(e -> outdatedExportIds.add(e.getKey()));
return outdatedExportIds;
}
private Blob storeNewBlob(String repositoryId) {
BlobStore store = createBlobStore(repositoryId);
if (!store.getAll().isEmpty()) {
store.clear();
}
return store.create(repositoryId);
}
private void storeExportInformation(String repositoryId, boolean withMetadata, boolean compressed, boolean encrypted) {
DataStore<RepositoryExportInformation> dataStore = createDataStore();
if (dataStore.get(repositoryId) != null) {
dataStore.remove(repositoryId);
}
String exporter = SecurityUtils.getSubject().getPrincipals().oneByType(User.class).getName();
RepositoryExportInformation info = new RepositoryExportInformation(exporter, Instant.now(), withMetadata, compressed, encrypted, ExportStatus.EXPORTING);
dataStore.put(repositoryId, info);
}
private Blob getBlob(String repositoryId) {
return createBlobStore(repositoryId).get(repositoryId);
}
private DataStore<RepositoryExportInformation> createDataStore() {
return dataStoreFactory.withType(RepositoryExportInformation.class).withName(STORE_NAME).build();
}
private BlobStore createBlobStore(String repositoryId) {
return blobStoreFactory.withName(STORE_NAME).forRepository(repositoryId).build();
}
}

View File

@@ -0,0 +1,31 @@
/*
* MIT License
*
* Copyright (c) 2020-present Cloudogu GmbH and Contributors
*
* Permission is hereby granted, free of charge, to any person obtaining a copy
* of this software and associated documentation files (the "Software"), to deal
* in the Software without restriction, including without limitation the rights
* to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
* copies of the Software, and to permit persons to whom the Software is
* furnished to do so, subject to the following conditions:
*
* The above copyright notice and this permission notice shall be included in all
* copies or substantial portions of the Software.
*
* THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
* IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
* FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
* AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
* LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
* OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
* SOFTWARE.
*/
package sonia.scm.importexport;
public enum ExportStatus {
EXPORTING,
INTERRUPTED,
FINISHED
}

View File

@@ -46,6 +46,7 @@ import java.io.OutputStream;
import java.nio.file.Files;
import java.nio.file.Paths;
public class FullScmRepositoryExporter {
static final String SCM_ENVIRONMENT_FILE_NAME = "scm-environment.xml";
@@ -57,6 +58,7 @@ public class FullScmRepositoryExporter {
private final TarArchiveRepositoryStoreExporter storeExporter;
private final WorkdirProvider workdirProvider;
private final RepositoryExportingCheck repositoryExportingCheck;
private final RepositoryImportExportEncryption repositoryImportExportEncryption;
@Inject
public FullScmRepositoryExporter(EnvironmentInformationXmlGenerator environmentGenerator,
@@ -64,28 +66,31 @@ public class FullScmRepositoryExporter {
RepositoryServiceFactory serviceFactory,
TarArchiveRepositoryStoreExporter storeExporter,
WorkdirProvider workdirProvider,
RepositoryExportingCheck repositoryExportingCheck) {
RepositoryExportingCheck repositoryExportingCheck,
RepositoryImportExportEncryption repositoryImportExportEncryption) {
this.environmentGenerator = environmentGenerator;
this.metadataGenerator = metadataGenerator;
this.serviceFactory = serviceFactory;
this.storeExporter = storeExporter;
this.workdirProvider = workdirProvider;
this.repositoryExportingCheck = repositoryExportingCheck;
this.repositoryImportExportEncryption = repositoryImportExportEncryption;
}
public void export(Repository repository, OutputStream outputStream) {
public void export(Repository repository, OutputStream outputStream, String password) {
repositoryExportingCheck.withExportingLock(repository, () -> {
exportInLock(repository, outputStream);
exportInLock(repository, outputStream, password);
return null;
});
}
private void exportInLock(Repository repository, OutputStream outputStream) {
private void exportInLock(Repository repository, OutputStream outputStream, String password) {
try (
RepositoryService service = serviceFactory.create(repository);
BufferedOutputStream bos = new BufferedOutputStream(outputStream);
GzipCompressorOutputStream gzos = new GzipCompressorOutputStream(bos);
TarArchiveOutputStream taos = Archives.createTarOutputStream(gzos)
OutputStream cos = repositoryImportExportEncryption.optionallyEncrypt(bos, password);
GzipCompressorOutputStream gzos = new GzipCompressorOutputStream(cos);
TarArchiveOutputStream taos = Archives.createTarOutputStream(gzos);
) {
writeEnvironmentData(taos);
writeMetadata(repository, taos);

View File

@@ -41,6 +41,7 @@ import java.io.InputStream;
import static java.util.Arrays.stream;
import static sonia.scm.util.Archives.createTarInputStream;
import static sonia.scm.ContextEntry.ContextBuilder.noContext;
public class FullScmRepositoryImporter {
@@ -48,24 +49,27 @@ public class FullScmRepositoryImporter {
private final ImportStep[] importSteps;
private final RepositoryManager repositoryManager;
private final RepositoryImportExportEncryption repositoryImportExportEncryption;
@Inject
public FullScmRepositoryImporter(EnvironmentCheckStep environmentCheckStep,
MetadataImportStep metadataImportStep,
StoreImportStep storeImportStep,
RepositoryImportStep repositoryImportStep,
RepositoryManager repositoryManager
) {
RepositoryManager repositoryManager,
RepositoryImportExportEncryption repositoryImportExportEncryption) {
this.repositoryManager = repositoryManager;
this.repositoryImportExportEncryption = repositoryImportExportEncryption;
importSteps = new ImportStep[]{environmentCheckStep, metadataImportStep, storeImportStep, repositoryImportStep};
}
public Repository importFromStream(Repository repository, InputStream inputStream) {
public Repository importFromStream(Repository repository, InputStream inputStream, String password) {
try {
if (inputStream.available() > 0) {
try (
BufferedInputStream bif = new BufferedInputStream(inputStream);
GzipCompressorInputStream gcis = new GzipCompressorInputStream(bif);
InputStream cif = repositoryImportExportEncryption.decrypt(bif, password);
GzipCompressorInputStream gcis = new GzipCompressorInputStream(cif);
TarArchiveInputStream tais = createTarInputStream(gcis)
) {
return run(repository, tais);
@@ -78,7 +82,7 @@ public class FullScmRepositoryImporter {
}
} catch (IOException e) {
throw new ImportFailedException(
ContextEntry.ContextBuilder.entity(repository).build(),
noContext(),
"Could not import repository data from stream; got io exception while reading",
e
);

View File

@@ -0,0 +1,55 @@
/*
* MIT License
*
* Copyright (c) 2020-present Cloudogu GmbH and Contributors
*
* Permission is hereby granted, free of charge, to any person obtaining a copy
* of this software and associated documentation files (the "Software"), to deal
* in the Software without restriction, including without limitation the rights
* to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
* copies of the Software, and to permit persons to whom the Software is
* furnished to do so, subject to the following conditions:
*
* The above copyright notice and this permission notice shall be included in all
* copies or substantial portions of the Software.
*
* THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
* IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
* FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
* AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
* LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
* OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
* SOFTWARE.
*/
package sonia.scm.importexport;
import lombok.AllArgsConstructor;
import lombok.Getter;
import lombok.NoArgsConstructor;
import lombok.Setter;
import sonia.scm.xml.XmlInstantAdapter;
import javax.xml.bind.annotation.XmlAccessType;
import javax.xml.bind.annotation.XmlAccessorType;
import javax.xml.bind.annotation.XmlRootElement;
import javax.xml.bind.annotation.adapters.XmlJavaTypeAdapter;
import java.time.Instant;
@NoArgsConstructor
@AllArgsConstructor
@Getter
@Setter
@XmlRootElement
@XmlAccessorType(XmlAccessType.FIELD)
public class RepositoryExportInformation {
private String exporterName;
@XmlJavaTypeAdapter(XmlInstantAdapter.class)
private Instant created;
private boolean withMetadata;
private boolean compressed;
private boolean encrypted;
private ExportStatus status;
}

View File

@@ -0,0 +1,195 @@
/*
* MIT License
*
* Copyright (c) 2020-present Cloudogu GmbH and Contributors
*
* Permission is hereby granted, free of charge, to any person obtaining a copy
* of this software and associated documentation files (the "Software"), to deal
* in the Software without restriction, including without limitation the rights
* to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
* copies of the Software, and to permit persons to whom the Software is
* furnished to do so, subject to the following conditions:
*
* The above copyright notice and this permission notice shall be included in all
* copies or substantial portions of the Software.
*
* THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
* IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
* FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
* AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
* LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
* OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
* SOFTWARE.
*/
package sonia.scm.importexport;
import com.google.common.base.Strings;
import javax.annotation.Nonnull;
import javax.annotation.Nullable;
import javax.crypto.Cipher;
import javax.crypto.CipherInputStream;
import javax.crypto.CipherOutputStream;
import javax.crypto.NoSuchPaddingException;
import javax.crypto.SecretKey;
import javax.crypto.SecretKeyFactory;
import javax.crypto.spec.IvParameterSpec;
import javax.crypto.spec.PBEKeySpec;
import javax.crypto.spec.SecretKeySpec;
import java.io.IOException;
import java.io.InputStream;
import java.io.OutputStream;
import java.nio.charset.Charset;
import java.nio.charset.StandardCharsets;
import java.security.InvalidAlgorithmParameterException;
import java.security.InvalidKeyException;
import java.security.NoSuchAlgorithmException;
import java.security.SecureRandom;
import java.security.spec.InvalidKeySpecException;
import java.security.spec.KeySpec;
import java.util.Arrays;
public class RepositoryImportExportEncryption {
private static final SecureRandom SECURE_RANDOM = new SecureRandom();
private static final String ENCRYPTION_HEADER = "SCMM_v1_";
private static final Charset ENCRYPTION_HEADER_CHARSET = StandardCharsets.ISO_8859_1;
static {
SECURE_RANDOM.setSeed(System.currentTimeMillis());
}
/**
* Returns an encrypting stream for the given origin stream, if a not-empty secret is given. Otherwise
* the original stream is returned. That is, this delegates to {@link #encrypt(OutputStream, String)} if
* a secret is given.
* @param origin The stream that should be encrypted, when a not-empty secret is given.
* @param secret The secret to use or <code>null</code> or an empty string, if no encryption should be used.
* @return An encrypted stream or <code>origin</code>, when no secret is given.
*/
public OutputStream optionallyEncrypt(OutputStream origin, @Nullable String secret) throws IOException {
if (!Strings.isNullOrEmpty(secret)) {
return encrypt(origin, secret);
} else {
return origin;
}
}
/**
* Encrypts the given stream with the given secret.
*/
public OutputStream encrypt(OutputStream origin, @Nonnull String secret) throws IOException {
byte[] salt = createSalt();
writeSaltHeader(origin, salt);
Cipher cipher = createCipher(Cipher.ENCRYPT_MODE, secret, salt);
return new CipherOutputStream(origin, cipher);
}
/**
* Returns a decrypting stream for the given input stream, if a not-empty secret is given. Otherwise
* the original stream is returned. That is, this delegated to {@link #decrypt(InputStream, String)} if
* a secret is given.
* @param stream The stream that should be decrypted, when a not-empty secret is given.
* @param secret The secret to use or <code>null</code> or an empty string, if no decryption should take place.
* @return A decrypted stream or <code>stream</code>, when no secret is given.
*/
public InputStream optionallyDecrypt(InputStream stream, @Nullable String secret) throws IOException {
if (!Strings.isNullOrEmpty(secret)) {
return decrypt(stream, secret);
} else {
return stream;
}
}
/**
* Decrypts the given stream with the given secret.
*/
public InputStream decrypt(InputStream encryptedStream, @Nonnull String secret) throws IOException {
byte[] salt = readSaltHeader(encryptedStream);
Cipher cipher = createCipher(Cipher.DECRYPT_MODE, secret, salt);
return new CipherInputStream(encryptedStream, cipher);
}
private void writeSaltHeader(OutputStream origin, byte[] salt) throws IOException {
origin.write(ENCRYPTION_HEADER.getBytes(ENCRYPTION_HEADER_CHARSET));
origin.write(salt);
}
private byte[] readSaltHeader(InputStream encryptedStream) throws IOException {
byte[] header = new byte[8];
int headerBytesRead = encryptedStream.read(header);
if (headerBytesRead != 8 || !ENCRYPTION_HEADER.equals(new String(header, ENCRYPTION_HEADER_CHARSET))) {
throw new IOException("Expected header with salt not found (\"Salted__\")");
}
byte[] salt = new byte[8];
int lengthRead = encryptedStream.read(salt);
if (lengthRead != 8) {
throw new IOException("Failed to read salt from input");
}
return salt;
}
private Cipher createCipher(int encryptMode, String key, byte[] salt) {
Cipher cipher = getCipher();
try {
cipher.init(encryptMode, getSecretKeySpec(key.toCharArray(), salt), getIvSpec(key.toCharArray(), salt));
} catch (InvalidKeyException | InvalidAlgorithmParameterException e) {
throw new IllegalStateException("Could not initialize cipher", e);
}
return cipher;
}
private Cipher getCipher() {
try {
return Cipher.getInstance("AES/CBC/PKCS5Padding");
} catch (NoSuchAlgorithmException | NoSuchPaddingException e) {
throw new IllegalStateException("Could not initialize cipher", e);
}
}
private byte[] createSalt() {
byte[] salt = new byte[8];
SECURE_RANDOM.nextBytes(salt);
return salt;
}
private SecretKeySpec getSecretKeySpec(char[] key, byte[] salt) {
SecretKey secretKey = computeSecretKey(key, salt);
return new SecretKeySpec(secretKey.getEncoded(), "AES");
}
private SecretKey computeSecretKey(char[] password, byte[] salt) {
KeySpec spec = getKeySpec(password, salt);
try {
return new SecretKeySpec(getSecretKeyFactory().generateSecret(spec).getEncoded(), "AES");
} catch (InvalidKeySpecException e) {
throw new IllegalStateException("could not create key spec", e);
}
}
private PBEKeySpec getKeySpec(char[] password, byte[] salt) {
return new PBEKeySpec(password, salt, 10000, 256);
}
@SuppressWarnings("java:S3329") // we generate the IV by deriving it from the password; this should be pseudo random enough
private IvParameterSpec getIvSpec(char[] password, byte[] salt) {
PBEKeySpec spec = new PBEKeySpec(password, salt, 1000, 256);
try {
byte[] bytes = getSecretKeyFactory().generateSecret(spec).getEncoded();
byte[] iv = Arrays.copyOfRange(bytes, 16, 16 + 16);
return new IvParameterSpec(iv);
} catch (InvalidKeySpecException e) {
throw new IllegalStateException("Could not derive from key", e);
}
}
private SecretKeyFactory getSecretKeyFactory() {
try {
return SecretKeyFactory.getInstance("PBKDF2WithHmacSHA256");
} catch (NoSuchAlgorithmException e) {
throw new IllegalStateException("Could not instantiate secret key factory", e);
}
}
}

View File

@@ -32,6 +32,7 @@ import sonia.scm.SCMContext;
import sonia.scm.config.ScmConfiguration;
import sonia.scm.group.Group;
import sonia.scm.group.GroupManager;
import sonia.scm.importexport.ExportService;
import sonia.scm.plugin.Extension;
import sonia.scm.security.AnonymousMode;
import sonia.scm.security.PermissionAssigner;
@@ -80,17 +81,19 @@ public class SetupContextListener implements ServletContextListener {
private final PermissionAssigner permissionAssigner;
private final ScmConfiguration scmConfiguration;
private final GroupManager groupManager;
private final ExportService exportService;
@VisibleForTesting
static final String AUTHENTICATED_GROUP_DESCRIPTION = "Includes all authenticated users";
@Inject
public SetupAction(UserManager userManager, PasswordService passwordService, PermissionAssigner permissionAssigner, ScmConfiguration scmConfiguration, GroupManager groupManager) {
public SetupAction(UserManager userManager, PasswordService passwordService, PermissionAssigner permissionAssigner, ScmConfiguration scmConfiguration, GroupManager groupManager, ExportService exportService) {
this.userManager = userManager;
this.passwordService = passwordService;
this.permissionAssigner = permissionAssigner;
this.scmConfiguration = scmConfiguration;
this.groupManager = groupManager;
this.exportService = exportService;
}
@Override
@@ -105,6 +108,8 @@ public class SetupContextListener implements ServletContextListener {
if (authenticatedGroupDoesNotExists()) {
createAuthenticatedGroup();
}
exportService.cleanupUnfinishedExports();
}
private boolean anonymousUserRequiredButNotExists() {

View File

@@ -0,0 +1,99 @@
/*
* MIT License
*
* Copyright (c) 2020-present Cloudogu GmbH and Contributors
*
* Permission is hereby granted, free of charge, to any person obtaining a copy
* of this software and associated documentation files (the "Software"), to deal
* in the Software without restriction, including without limitation the rights
* to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
* copies of the Software, and to permit persons to whom the Software is
* furnished to do so, subject to the following conditions:
*
* The above copyright notice and this permission notice shall be included in all
* copies or substantial portions of the Software.
*
* THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
* IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
* FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
* AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
* LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
* OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
* SOFTWARE.
*/
package sonia.scm.api.v2.resources;
import org.junit.jupiter.api.BeforeEach;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.extension.ExtendWith;
import org.mockito.Answers;
import org.mockito.InjectMocks;
import org.mockito.Mock;
import org.mockito.junit.jupiter.MockitoExtension;
import sonia.scm.importexport.ExportService;
import sonia.scm.importexport.ExportStatus;
import sonia.scm.importexport.RepositoryExportInformation;
import sonia.scm.repository.Repository;
import sonia.scm.repository.RepositoryTestData;
import java.net.URI;
import java.time.Instant;
import static org.assertj.core.api.Assertions.assertThat;
import static org.mockito.Mockito.when;
@ExtendWith(MockitoExtension.class)
class RepositoryExportInformationToDtoMapperTest {
private static final Repository REPOSITORY = RepositoryTestData.createHeartOfGold();
private final ResourceLinks resourceLinks = ResourceLinksMock.createMock(URI.create("/scm/api/"));
@Mock(answer = Answers.RETURNS_DEEP_STUBS)
private ExportService exportService;
private RepositoryExportInformationToDtoMapperImpl mapper;
@BeforeEach
void initResourceLinks() {
mapper = new RepositoryExportInformationToDtoMapperImpl();
mapper.setResourceLinks(resourceLinks);
mapper.setExportService(exportService);
}
@Test
void shouldMapToInfoDtoWithLinks() {
when(exportService.isExporting(REPOSITORY)).thenReturn(false);
when(exportService.getExportInformation(REPOSITORY).getStatus()).thenReturn(ExportStatus.FINISHED);
String exporterName = "trillian";
Instant now = Instant.now();
RepositoryExportInformation info = new RepositoryExportInformation(exporterName, now, true, true, false, ExportStatus.FINISHED);
RepositoryExportInformationDto dto = mapper.map(info, REPOSITORY);
assertThat(dto.getExporterName()).isEqualTo(exporterName);
assertThat(dto.getCreated()).isEqualTo(now);
assertThat(dto.isCompressed()).isTrue();
assertThat(dto.isWithMetadata()).isTrue();
assertThat(dto.isEncrypted()).isFalse();
assertThat(dto.getStatus()).isEqualTo(ExportStatus.FINISHED);
assertThat(dto.getLinks().getLinkBy("self").get().getHref()).isEqualTo("/scm/api/v2/repositories/hitchhiker/HeartOfGold/export/info");
assertThat(dto.getLinks().getLinkBy("download").get().getHref()).isEqualTo("/scm/api/v2/repositories/hitchhiker/HeartOfGold/export/download");
}
@Test
void shouldNotAppendDownloadLink() {
when(exportService.isExporting(REPOSITORY)).thenReturn(true);
String exporterName = "trillian";
Instant now = Instant.now();
RepositoryExportInformation info = new RepositoryExportInformation(exporterName, now, true, true, false, ExportStatus.EXPORTING);
RepositoryExportInformationDto dto = mapper.map(info, REPOSITORY);
assertThat(dto.getLinks().getLinkBy("self").get().getHref()).isEqualTo("/scm/api/v2/repositories/hitchhiker/HeartOfGold/export/info");
assertThat(dto.getLinks().getLinkBy("download")).isNotPresent();
}
}

View File

@@ -41,11 +41,16 @@ import org.mockito.ArgumentCaptor;
import org.mockito.Captor;
import org.mockito.InjectMocks;
import org.mockito.Mock;
import sonia.scm.NotFoundException;
import sonia.scm.PageResult;
import sonia.scm.config.ScmConfiguration;
import sonia.scm.event.ScmEventBus;
import sonia.scm.importexport.ExportFileExtensionResolver;
import sonia.scm.importexport.ExportService;
import sonia.scm.importexport.ExportStatus;
import sonia.scm.importexport.FullScmRepositoryExporter;
import sonia.scm.importexport.FullScmRepositoryImporter;
import sonia.scm.importexport.RepositoryImportExportEncryption;
import sonia.scm.repository.CustomNamespaceStrategy;
import sonia.scm.repository.NamespaceAndName;
import sonia.scm.repository.NamespaceStrategy;
@@ -82,16 +87,17 @@ import java.io.UnsupportedEncodingException;
import java.net.URI;
import java.net.URISyntaxException;
import java.net.URL;
import java.time.Instant;
import java.util.Collections;
import java.util.Map;
import java.util.Set;
import java.util.UUID;
import java.util.function.Consumer;
import java.util.function.Predicate;
import java.util.function.Supplier;
import static java.util.Collections.singletonList;
import static java.util.stream.Stream.of;
import static javax.servlet.http.HttpServletResponse.SC_ACCEPTED;
import static javax.servlet.http.HttpServletResponse.SC_BAD_REQUEST;
import static javax.servlet.http.HttpServletResponse.SC_CONFLICT;
import static javax.servlet.http.HttpServletResponse.SC_CREATED;
@@ -108,7 +114,6 @@ import static org.mockito.ArgumentMatchers.anyMap;
import static org.mockito.ArgumentMatchers.anyString;
import static org.mockito.ArgumentMatchers.eq;
import static org.mockito.Mockito.RETURNS_SELF;
import static org.mockito.Mockito.doAnswer;
import static org.mockito.Mockito.doReturn;
import static org.mockito.Mockito.doThrow;
import static org.mockito.Mockito.mock;
@@ -155,7 +160,15 @@ public class RepositoryRootResourceTest extends RepositoryTestBase {
@Mock
private FullScmRepositoryExporter fullScmRepositoryExporter;
@Mock
private RepositoryExportInformationToDtoMapper exportInformationToDtoMapper;
@Mock
private FullScmRepositoryImporter fullScmRepositoryImporter;
@Mock
private RepositoryImportExportEncryption repositoryImportExportEncryption;
@Mock
private ExportFileExtensionResolver fileExtensionResolver;
@Mock
private ExportService exportService;
@Captor
private ArgumentCaptor<Predicate<Repository>> filterCaptor;
@@ -170,15 +183,15 @@ public class RepositoryRootResourceTest extends RepositoryTestBase {
private RepositoryDtoToRepositoryMapperImpl dtoToRepositoryMapper;
@Before
public void prepareEnvironment() {
public void prepareEnvironment() throws IOException {
openMocks(this);
super.repositoryToDtoMapper = repositoryToDtoMapper;
super.dtoToRepositoryMapper = dtoToRepositoryMapper;
super.manager = repositoryManager;
RepositoryCollectionToDtoMapper repositoryCollectionToDtoMapper = new RepositoryCollectionToDtoMapper(repositoryToDtoMapper, resourceLinks);
super.repositoryCollectionResource = new RepositoryCollectionResource(repositoryManager, repositoryCollectionToDtoMapper, dtoToRepositoryMapper, resourceLinks, repositoryInitializer);
super.repositoryImportResource = new RepositoryImportResource(repositoryManager, dtoToRepositoryMapper, serviceFactory, resourceLinks, eventBus, fullScmRepositoryImporter);
super.repositoryExportResource = new RepositoryExportResource(repositoryManager, serviceFactory, fullScmRepositoryExporter);
super.repositoryImportResource = new RepositoryImportResource(repositoryManager, dtoToRepositoryMapper, serviceFactory, resourceLinks, eventBus, fullScmRepositoryImporter, repositoryImportExportEncryption);
super.repositoryExportResource = new RepositoryExportResource(repositoryManager, serviceFactory, fullScmRepositoryExporter, repositoryImportExportEncryption, exportService, exportInformationToDtoMapper, fileExtensionResolver, resourceLinks);
dispatcher.addSingletonResource(getRepositoryRootResource());
when(serviceFactory.create(any(Repository.class))).thenReturn(service);
when(scmPathInfoStore.get()).thenReturn(uriInfo);
@@ -191,6 +204,7 @@ public class RepositoryRootResourceTest extends RepositoryTestBase {
.principals(trillian)
.authenticated(true)
.buildSubject());
when(repositoryImportExportEncryption.optionallyEncrypt(any(), any())).thenAnswer(invocation -> invocation.getArgument(0));
}
@Test
@@ -543,12 +557,12 @@ public class RepositoryRootResourceTest extends RepositoryTestBase {
when(service.getPullCommand()).thenReturn(pullCommandBuilder);
Repository repository = RepositoryTestData.createHeartOfGold();
RepositoryImportResource.RepositoryImportDto repositoryImportDto = new RepositoryImportResource.RepositoryImportDto();
repositoryImportDto.setImportUrl("https://scm-manager.org/scm/repo/scmadmin/scm-manager.git");
repositoryImportDto.setNamespace("scmadmin");
repositoryImportDto.setName("scm-manager");
RepositoryImportResource.RepositoryImportFromUrlDto repositoryImportFromUrlDto = new RepositoryImportResource.RepositoryImportFromUrlDto();
repositoryImportFromUrlDto.setImportUrl("https://scm-manager.org/scm/repo/scmadmin/scm-manager.git");
repositoryImportFromUrlDto.setNamespace("scmadmin");
repositoryImportFromUrlDto.setName("scm-manager");
Consumer<Repository> repositoryConsumer = repositoryImportResource.pullChangesFromRemoteUrl(repositoryImportDto);
Consumer<Repository> repositoryConsumer = repositoryImportResource.pullChangesFromRemoteUrl(repositoryImportFromUrlDto);
repositoryConsumer.accept(repository);
verify(pullCommandBuilder).pull("https://scm-manager.org/scm/repo/scmadmin/scm-manager.git");
@@ -560,14 +574,14 @@ public class RepositoryRootResourceTest extends RepositoryTestBase {
when(service.getPullCommand()).thenReturn(pullCommandBuilder);
Repository repository = RepositoryTestData.createHeartOfGold();
RepositoryImportResource.RepositoryImportDto repositoryImportDto = new RepositoryImportResource.RepositoryImportDto();
repositoryImportDto.setImportUrl("https://scm-manager.org/scm/repo/scmadmin/scm-manager.git");
repositoryImportDto.setNamespace("scmadmin");
repositoryImportDto.setName("scm-manager");
repositoryImportDto.setUsername("trillian");
repositoryImportDto.setPassword("secret");
RepositoryImportResource.RepositoryImportFromUrlDto repositoryImportFromUrlDto = new RepositoryImportResource.RepositoryImportFromUrlDto();
repositoryImportFromUrlDto.setImportUrl("https://scm-manager.org/scm/repo/scmadmin/scm-manager.git");
repositoryImportFromUrlDto.setNamespace("scmadmin");
repositoryImportFromUrlDto.setName("scm-manager");
repositoryImportFromUrlDto.setUsername("trillian");
repositoryImportFromUrlDto.setPassword("secret");
Consumer<Repository> repositoryConsumer = repositoryImportResource.pullChangesFromRemoteUrl(repositoryImportDto);
Consumer<Repository> repositoryConsumer = repositoryImportResource.pullChangesFromRemoteUrl(repositoryImportFromUrlDto);
repositoryConsumer.accept(repository);
verify(pullCommandBuilder).withUsername("trillian");
@@ -581,12 +595,12 @@ public class RepositoryRootResourceTest extends RepositoryTestBase {
doThrow(ImportFailedException.class).when(pullCommandBuilder).pull(anyString());
Repository repository = RepositoryTestData.createHeartOfGold();
RepositoryImportResource.RepositoryImportDto repositoryImportDto = new RepositoryImportResource.RepositoryImportDto();
repositoryImportDto.setImportUrl("https://scm-manager.org/scm/repo/scmadmin/scm-manager.git");
repositoryImportDto.setNamespace("scmadmin");
repositoryImportDto.setName("scm-manager");
RepositoryImportResource.RepositoryImportFromUrlDto repositoryImportFromUrlDto = new RepositoryImportResource.RepositoryImportFromUrlDto();
repositoryImportFromUrlDto.setImportUrl("https://scm-manager.org/scm/repo/scmadmin/scm-manager.git");
repositoryImportFromUrlDto.setNamespace("scmadmin");
repositoryImportFromUrlDto.setName("scm-manager");
Consumer<Repository> repositoryConsumer = repositoryImportResource.pullChangesFromRemoteUrl(repositoryImportDto);
Consumer<Repository> repositoryConsumer = repositoryImportResource.pullChangesFromRemoteUrl(repositoryImportFromUrlDto);
assertThrows(ImportFailedException.class, () -> repositoryConsumer.accept(repository));
}
@@ -732,6 +746,7 @@ public class RepositoryRootResourceTest extends RepositoryTestBase {
BundleCommandBuilder bundleCommandBuilder = mock(BundleCommandBuilder.class);
when(service.getBundleCommand()).thenReturn(bundleCommandBuilder);
when(bundleCommandBuilder.getFileExtension()).thenReturn(".bundle");
MockHttpRequest request = MockHttpRequest
.get("/" + RepositoryRootResource.REPOSITORIES_PATH_V2 + "space/repo/export/svn");
@@ -754,6 +769,7 @@ public class RepositoryRootResourceTest extends RepositoryTestBase {
BundleCommandBuilder bundleCommandBuilder = mock(BundleCommandBuilder.class);
when(service.getBundleCommand()).thenReturn(bundleCommandBuilder);
when(bundleCommandBuilder.getFileExtension()).thenReturn(".bundle");
MockHttpRequest request = MockHttpRequest
.get("/" + RepositoryRootResource.REPOSITORIES_PATH_V2 + "space/repo/export/svn?compressed=true");
@@ -785,7 +801,206 @@ public class RepositoryRootResourceTest extends RepositoryTestBase {
assertEquals(SC_OK, response.getStatus());
assertEquals("application/x-gzip", response.getOutputHeaders().get("Content-Type").get(0).toString());
verify(fullScmRepositoryExporter).export(eq(repository), any(OutputStream.class));
verify(fullScmRepositoryExporter).export(eq(repository), any(OutputStream.class), any());
}
@Test
public void shouldExportFullRepositoryWithPassword() throws URISyntaxException {
String namespace = "space";
String name = "repo";
Repository repository = createRepository(namespace, name, "svn");
when(manager.get(new NamespaceAndName(namespace, name))).thenReturn(repository);
mockRepositoryHandler(ImmutableSet.of(Command.BUNDLE));
BundleCommandBuilder bundleCommandBuilder = mock(BundleCommandBuilder.class);
when(service.getBundleCommand()).thenReturn(bundleCommandBuilder);
MockHttpRequest request = MockHttpRequest
.post("/" + RepositoryRootResource.REPOSITORIES_PATH_V2 + "space/repo/export/full")
.contentType(VndMediaType.REPOSITORY_EXPORT)
.content("{\"password\": \"hitchhiker\"}".getBytes());
MockHttpResponse response = new MockHttpResponse();
dispatcher.invoke(request, response);
assertEquals(SC_OK, response.getStatus());
assertEquals("application/x-gzip", response.getOutputHeaders().get("Content-Type").get(0).toString());
verify(fullScmRepositoryExporter).export(eq(repository), any(OutputStream.class), any());
}
@Test
public void shouldExportFullRepositoryAsyncWithPassword() throws URISyntaxException {
String namespace = "space";
String name = "repo";
Repository repository = createRepository(namespace, name, "svn");
when(manager.get(new NamespaceAndName(namespace, name))).thenReturn(repository);
mockRepositoryHandler(ImmutableSet.of(Command.BUNDLE));
BundleCommandBuilder bundleCommandBuilder = mock(BundleCommandBuilder.class);
when(service.getBundleCommand()).thenReturn(bundleCommandBuilder);
MockHttpRequest request = MockHttpRequest
.post("/" + RepositoryRootResource.REPOSITORIES_PATH_V2 + "space/repo/export/full")
.contentType(VndMediaType.REPOSITORY_EXPORT)
.content("{\"password\": \"hitchhiker\", \"async\":\"true\"}".getBytes());
MockHttpResponse response = new MockHttpResponse();
dispatcher.invoke(request, response);
assertEquals(SC_ACCEPTED, response.getStatus());
assertEquals("/v2/repositories/space/repo/export/download", response.getOutputHeaders().getFirst("SCM-Export-Download"));
}
@Test
public void shouldReturnConflictIfRepositoryAlreadyExporting() throws URISyntaxException {
String namespace = "space";
String name = "repo";
Repository repository = createRepository(namespace, name, "svn");
when(manager.get(new NamespaceAndName(namespace, name))).thenReturn(repository);
mockRepositoryHandler(ImmutableSet.of(Command.BUNDLE));
BundleCommandBuilder bundleCommandBuilder = mock(BundleCommandBuilder.class);
when(service.getBundleCommand()).thenReturn(bundleCommandBuilder);
when(exportService.isExporting(repository)).thenReturn(true);
MockHttpRequest request = MockHttpRequest
.post("/" + RepositoryRootResource.REPOSITORIES_PATH_V2 + "space/repo/export/full")
.contentType(VndMediaType.REPOSITORY_EXPORT)
.content("{\"password\": \"hitchhiker\", \"async\":\"true\"}".getBytes());
MockHttpResponse response = new MockHttpResponse();
dispatcher.invoke(request, response);
assertEquals(SC_CONFLICT, response.getStatus());
}
@Test
public void shouldDeleteRepositoryExport() throws URISyntaxException, IOException {
String namespace = "space";
String name = "repo";
Repository repository = createRepository(namespace, name, "svn");
when(manager.get(new NamespaceAndName(namespace, name))).thenReturn(repository);
mockRepositoryHandler(ImmutableSet.of(Command.BUNDLE));
BundleCommandBuilder bundleCommandBuilder = mock(BundleCommandBuilder.class);
when(service.getBundleCommand()).thenReturn(bundleCommandBuilder);
when(exportService.isExporting(repository)).thenReturn(false);
when(exportService.getData(repository)).thenReturn(new ByteArrayInputStream("".getBytes()));
MockHttpRequest request = MockHttpRequest
.delete("/" + RepositoryRootResource.REPOSITORIES_PATH_V2 + "space/repo/export");
MockHttpResponse response = new MockHttpResponse();
dispatcher.invoke(request, response);
assertEquals(SC_NO_CONTENT, response.getStatus());
verify(exportService).clear(repository.getId());
}
@Test
public void shouldReturnNotFoundIfExportDoesNotExist() throws URISyntaxException {
String namespace = "space";
String name = "repo";
Repository repository = createRepository(namespace, name, "svn");
when(manager.get(new NamespaceAndName(namespace, name))).thenReturn(repository);
mockRepositoryHandler(ImmutableSet.of(Command.BUNDLE));
BundleCommandBuilder bundleCommandBuilder = mock(BundleCommandBuilder.class);
when(service.getBundleCommand()).thenReturn(bundleCommandBuilder);
when(exportService.isExporting(repository)).thenReturn(false);
doThrow(NotFoundException.class).when(exportService).checkExportIsAvailable(repository);
MockHttpRequest request = MockHttpRequest
.get("/" + RepositoryRootResource.REPOSITORIES_PATH_V2 + "space/repo/export/download");
MockHttpResponse response = new MockHttpResponse();
dispatcher.invoke(request, response);
assertEquals(SC_NOT_FOUND, response.getStatus());
verify(exportService).checkExportIsAvailable(repository);
}
@Test
public void shouldReturnConflictIfExportIsStillExporting() throws URISyntaxException {
String namespace = "space";
String name = "repo";
Repository repository = createRepository(namespace, name, "svn");
when(manager.get(new NamespaceAndName(namespace, name))).thenReturn(repository);
mockRepositoryHandler(ImmutableSet.of(Command.BUNDLE));
BundleCommandBuilder bundleCommandBuilder = mock(BundleCommandBuilder.class);
when(service.getBundleCommand()).thenReturn(bundleCommandBuilder);
when(exportService.isExporting(repository)).thenReturn(true);
MockHttpRequest request = MockHttpRequest
.get("/" + RepositoryRootResource.REPOSITORIES_PATH_V2 + "space/repo/export/download");
MockHttpResponse response = new MockHttpResponse();
dispatcher.invoke(request, response);
assertEquals(SC_CONFLICT, response.getStatus());
}
@Test
public void shouldDownloadRepositoryExportIfReady() throws URISyntaxException, IOException {
String namespace = "space";
String name = "repo";
Repository repository = createRepository(namespace, name, "svn");
when(manager.get(new NamespaceAndName(namespace, name))).thenReturn(repository);
mockRepositoryHandler(ImmutableSet.of(Command.BUNDLE));
BundleCommandBuilder bundleCommandBuilder = mock(BundleCommandBuilder.class);
when(service.getBundleCommand()).thenReturn(bundleCommandBuilder);
when(exportService.isExporting(repository)).thenReturn(false);
when(exportService.getData(repository)).thenReturn(new ByteArrayInputStream("content".getBytes()));
when(exportService.getFileExtension(repository)).thenReturn("tar.gz.enc");
MockHttpRequest request = MockHttpRequest
.get("/" + RepositoryRootResource.REPOSITORIES_PATH_V2 + "space/repo/export/download");
MockHttpResponse response = new MockHttpResponse();
dispatcher.invoke(request, response);
assertEquals(SC_OK, response.getStatus());
verify(exportService).getData(repository);
}
@Test
public void shouldReturnExportInfo() throws URISyntaxException, IOException {
String namespace = "space";
String name = "repo";
Repository repository = createRepository(namespace, name, "svn");
when(manager.get(new NamespaceAndName(namespace, name))).thenReturn(repository);
mockRepositoryHandler(ImmutableSet.of(Command.BUNDLE));
BundleCommandBuilder bundleCommandBuilder = mock(BundleCommandBuilder.class);
when(service.getBundleCommand()).thenReturn(bundleCommandBuilder);
RepositoryExportInformationDto dto = new RepositoryExportInformationDto();
dto.setExporterName("trillian");
dto.setCreated(Instant.ofEpochMilli(100));
dto.setStatus(ExportStatus.EXPORTING);
when(exportInformationToDtoMapper.map(any(), eq(repository))).thenReturn(dto);
MockHttpRequest request = MockHttpRequest
.get("/" + RepositoryRootResource.REPOSITORIES_PATH_V2 + "space/repo/export/info");
MockHttpResponse response = new MockHttpResponse();
dispatcher.invoke(request, response);
assertEquals(
"{\"exporterName\":\"trillian\",\"created\":0.100000000,\"withMetadata"
+ "\":false,\"compressed\":false,\"encrypted\":false,\"status\":\"EXPORTING\"}",
response.getContentAsString()
);
assertEquals(SC_OK, response.getStatus());
verify(exportService).getExportInformation(repository);
}
private void mockRepositoryHandler(Set<Command> cmds) {

View File

@@ -285,23 +285,19 @@ public class RepositoryToRepositoryDtoMapperTest {
}
@Test
public void shouldCreateExportLink() {
public void shouldCreateExportLinks() {
Repository repository = createTestRepository();
repository.setType("svn");
RepositoryDto dto = mapper.map(repository);
assertEquals(
"http://example.com/base/v2/repositories/testspace/test/export/svn",
dto.getLinks().getLinkBy("export").get().getHref());
}
@Test
public void shouldCreateFullExportLink() {
Repository repository = createTestRepository();
repository.setType("svn");
RepositoryDto dto = mapper.map(repository);
assertEquals(
"http://example.com/base/v2/repositories/testspace/test/export/full",
dto.getLinks().getLinkBy("fullExport").get().getHref());
assertEquals(
"http://example.com/base/v2/repositories/testspace/test/export/info",
dto.getLinks().getLinkBy("exportInfo").get().getHref());
}
private ScmProtocol mockProtocol(String type, String protocol) {

View File

@@ -0,0 +1,113 @@
/*
* MIT License
*
* Copyright (c) 2020-present Cloudogu GmbH and Contributors
*
* Permission is hereby granted, free of charge, to any person obtaining a copy
* of this software and associated documentation files (the "Software"), to deal
* in the Software without restriction, including without limitation the rights
* to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
* copies of the Software, and to permit persons to whom the Software is
* furnished to do so, subject to the following conditions:
*
* The above copyright notice and this permission notice shall be included in all
* copies or substantial portions of the Software.
*
* THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
* IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
* FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
* AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
* LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
* OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
* SOFTWARE.
*/
package sonia.scm.importexport;
import org.junit.jupiter.api.BeforeEach;
import org.junit.jupiter.api.Nested;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.extension.ExtendWith;
import org.mockito.InjectMocks;
import org.mockito.Mock;
import org.mockito.junit.jupiter.MockitoExtension;
import sonia.scm.repository.Repository;
import sonia.scm.repository.RepositoryTestData;
import sonia.scm.repository.api.BundleCommandBuilder;
import sonia.scm.repository.api.RepositoryService;
import sonia.scm.repository.api.RepositoryServiceFactory;
import sonia.scm.repository.spi.BundleCommand;
import sonia.scm.user.User;
import static org.assertj.core.api.Assertions.assertThat;
import static org.mockito.Mockito.mock;
import static org.mockito.Mockito.when;
@ExtendWith(MockitoExtension.class)
class ExportFileExtensionResolverTest {
private static final Repository REPOSITORY = RepositoryTestData.createHeartOfGold();
@Mock
private RepositoryServiceFactory serviceFactory;
@Mock
private RepositoryService service;
@Mock
private BundleCommandBuilder bundleCommand;
@InjectMocks
private ExportFileExtensionResolver resolver;
@Test
void shouldResolveWithMetadata() {
String result = resolver.resolve(REPOSITORY, true, false, false);
assertThat(result).isEqualTo("tar.gz");
}
@Test
void shouldResolveWithMetadataAndEncrypted() {
String result = resolver.resolve(REPOSITORY, true, false, true);
assertThat(result).isEqualTo("tar.gz.enc");
}
@Nested
class withRepositoryService {
@BeforeEach
void initBundleCommand() {
when(serviceFactory.create(REPOSITORY)).thenReturn(service);
when(service.getBundleCommand()).thenReturn(bundleCommand);
when(bundleCommand.getFileExtension()).thenReturn("dump");
}
@Test
void shouldResolveDump() {
String result = resolver.resolve(REPOSITORY, false, false, false);
assertThat(result).isEqualTo("dump");
}
@Test
void shouldResolveDump_Compressed() {
String result = resolver.resolve(REPOSITORY, false, true, false);
assertThat(result).isEqualTo("dump.gz");
}
@Test
void shouldResolveDump_Encrypted() {
String result = resolver.resolve(REPOSITORY, false, false, true);
assertThat(result).isEqualTo("dump.enc");
}
@Test
void shouldResolveDump_Compressed_Encrypted() {
String result = resolver.resolve(REPOSITORY, false, true, true);
assertThat(result).isEqualTo("dump.gz.enc");
}
}
}

View File

@@ -0,0 +1,243 @@
/*
* MIT License
*
* Copyright (c) 2020-present Cloudogu GmbH and Contributors
*
* Permission is hereby granted, free of charge, to any person obtaining a copy
* of this software and associated documentation files (the "Software"), to deal
* in the Software without restriction, including without limitation the rights
* to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
* copies of the Software, and to permit persons to whom the Software is
* furnished to do so, subject to the following conditions:
*
* The above copyright notice and this permission notice shall be included in all
* copies or substantial portions of the Software.
*
* THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
* IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
* FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
* AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
* LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
* OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
* SOFTWARE.
*/
package sonia.scm.importexport;
import org.apache.shiro.subject.PrincipalCollection;
import org.apache.shiro.subject.Subject;
import org.apache.shiro.util.ThreadContext;
import org.junit.jupiter.api.AfterEach;
import org.junit.jupiter.api.BeforeEach;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.extension.ExtendWith;
import org.mockito.Answers;
import org.mockito.InjectMocks;
import org.mockito.Mock;
import org.mockito.junit.jupiter.MockitoExtension;
import sonia.scm.NotFoundException;
import sonia.scm.repository.Repository;
import sonia.scm.repository.RepositoryTestData;
import sonia.scm.store.Blob;
import sonia.scm.store.BlobStore;
import sonia.scm.store.BlobStoreFactory;
import sonia.scm.store.DataStore;
import sonia.scm.store.DataStoreFactory;
import sonia.scm.store.InMemoryBlobStore;
import sonia.scm.store.InMemoryDataStore;
import sonia.scm.user.User;
import java.io.IOException;
import java.io.OutputStream;
import java.time.Instant;
import java.time.temporal.ChronoUnit;
import java.util.List;
import static org.assertj.core.api.Assertions.assertThat;
import static org.junit.jupiter.api.Assertions.assertThrows;
import static org.mockito.Mockito.doNothing;
import static org.mockito.Mockito.lenient;
import static org.mockito.Mockito.mock;
import static org.mockito.Mockito.when;
import static sonia.scm.importexport.ExportService.STORE_NAME;
@ExtendWith(MockitoExtension.class)
class ExportServiceTest {
private static final Repository REPOSITORY = RepositoryTestData.createHeartOfGold();
@Mock(answer = Answers.RETURNS_DEEP_STUBS)
private BlobStoreFactory blobStoreFactory;
@Mock(answer = Answers.RETURNS_DEEP_STUBS)
private DataStoreFactory dataStoreFactory;
@Mock
private ExportFileExtensionResolver resolver;
private BlobStore blobStore;
private DataStore<RepositoryExportInformation> dataStore;
@Mock
private Subject subject;
@InjectMocks
private ExportService exportService;
@BeforeEach
void initMocks() {
ThreadContext.bind(subject);
PrincipalCollection principalCollection = mock(PrincipalCollection.class);
lenient().when(subject.getPrincipals()).thenReturn(principalCollection);
lenient().when(principalCollection.oneByType(User.class)).thenReturn(
new User("trillian", "Trillian", "trillian@hitchhiker.org")
);
blobStore = new InMemoryBlobStore();
when(blobStoreFactory.withName(STORE_NAME).forRepository(REPOSITORY.getId()).build())
.thenReturn(blobStore);
dataStore = new InMemoryDataStore<>();
when(dataStoreFactory.withType(RepositoryExportInformation.class).withName(STORE_NAME).build())
.thenReturn(dataStore);
}
@AfterEach
void tearDown() {
ThreadContext.unbindSubject();
}
@Test
void shouldClearStoreIfEntryAlreadyExists() throws IOException {
//Old content blob
blobStore.create(REPOSITORY.getId());
String newContent = "Scm-Manager-Export";
OutputStream os = exportService.store(REPOSITORY, true, true, true);
os.write(newContent.getBytes());
os.flush();
os.close();
// Only new blob should exist
List<Blob> blobs = blobStore.getAll();
assertThat(blobs).hasSize(1);
//Verify content
byte[] bytes = new byte[18];
exportService.getData(REPOSITORY).read(bytes);
assertThat(new String(bytes)).isEqualTo(newContent);
}
@Test
void shouldShowCorrectExportStatus() {
doNothing().when(subject).checkPermission("repository:export:" + REPOSITORY.getId());
exportService.store(REPOSITORY, false, false, false);
assertThat(exportService.isExporting(REPOSITORY)).isTrue();
exportService.setExportFinished(REPOSITORY);
assertThat(exportService.isExporting(REPOSITORY)).isFalse();
}
@Test
void shouldOnlyClearRepositoryExports() {
doNothing().when(subject).checkPermission("repository:export:" + REPOSITORY.getId());
Repository hvpt = RepositoryTestData.createHappyVerticalPeopleTransporter();
dataStore.put(hvpt.getId(), new RepositoryExportInformation());
blobStore.create(REPOSITORY.getId());
dataStore.put(REPOSITORY.getId(), new RepositoryExportInformation());
exportService.clear(REPOSITORY.getId());
assertThat(dataStore.get(REPOSITORY.getId())).isNull();
assertThat(dataStore.get(hvpt.getId())).isNotNull();
assertThat(blobStore.getAll()).isEmpty();
}
@Test
void shouldGetExportInformation() {
doNothing().when(subject).checkPermission("repository:export:" + REPOSITORY.getId());
exportService.store(REPOSITORY, true, true, false);
RepositoryExportInformation exportInformation = exportService.getExportInformation(REPOSITORY);
assertThat(exportInformation.getExporterName()).isEqualTo("trillian");
assertThat(exportInformation.getCreated()).isNotNull();
}
@Test
void shouldThrowNotFoundException() {
assertThrows(NotFoundException.class, () -> exportService.getExportInformation(REPOSITORY));
assertThrows(NotFoundException.class, () -> exportService.getFileExtension(REPOSITORY));
assertThrows(NotFoundException.class, () -> exportService.getData(REPOSITORY));
}
@Test
void shouldResolveFileExtension() {
doNothing().when(subject).checkPermission("repository:export:" + REPOSITORY.getId());
String extension = "tar.gz.enc";
RepositoryExportInformation info = new RepositoryExportInformation();
dataStore.put(REPOSITORY.getId(), info);
when(resolver.resolve(REPOSITORY, false, false, false)).thenReturn(extension);
String fileExtension = exportService.getFileExtension(REPOSITORY);
assertThat(fileExtension).isEqualTo(extension);
}
@Test
void shouldOnlyCleanupUnfinishedExports() {
blobStore.create(REPOSITORY.getId());
RepositoryExportInformation info = new RepositoryExportInformation();
info.setStatus(ExportStatus.EXPORTING);
dataStore.put(
REPOSITORY.getId(),
info
);
Repository finishedExport = RepositoryTestData.createHappyVerticalPeopleTransporter();
BlobStore finishedExportBlobStore = new InMemoryBlobStore();
Blob finishedExportBlob = finishedExportBlobStore.create(finishedExport.getId());
RepositoryExportInformation finishedExportInfo = new RepositoryExportInformation();
finishedExportInfo.setStatus(ExportStatus.FINISHED);
dataStore.put(
finishedExport.getId(),
finishedExportInfo
);
when(blobStoreFactory.withName(STORE_NAME).forRepository(finishedExport.getId()).build())
.thenReturn(finishedExportBlobStore);
exportService.cleanupUnfinishedExports();
assertThat(blobStore.getAll()).isEmpty();
assertThat(dataStore.get(REPOSITORY.getId()).getStatus()).isEqualTo(ExportStatus.INTERRUPTED);
assertThat(finishedExportBlobStore.get(finishedExport.getId())).isEqualTo(finishedExportBlob);
assertThat(dataStore.get(finishedExport.getId()).getStatus()).isEqualTo(ExportStatus.FINISHED);
}
@Test
void shouldOnlyCleanupOutdatedExports() {
blobStore.create(REPOSITORY.getId());
Instant now = Instant.now();
RepositoryExportInformation newExportInfo = new RepositoryExportInformation();
newExportInfo.setCreated(now);
dataStore.put(REPOSITORY.getId(), newExportInfo);
Repository oldExportRepo = RepositoryTestData.createHappyVerticalPeopleTransporter();
BlobStore oldExportBlobStore = new InMemoryBlobStore();
oldExportBlobStore.create(oldExportRepo.getId());
RepositoryExportInformation oldExportInfo = new RepositoryExportInformation();
Instant old = Instant.now().minus(11, ChronoUnit.DAYS);
oldExportInfo.setCreated(old);
dataStore.put(oldExportRepo.getId(), oldExportInfo);
when(blobStoreFactory.withName(STORE_NAME).forRepository(oldExportRepo.getId()).build())
.thenReturn(oldExportBlobStore);
exportService.cleanupOutdatedExports();
assertThat(blobStore.getAll()).hasSize(1);
assertThat(oldExportBlobStore.getAll()).isEmpty();
assertThat(dataStore.get(REPOSITORY.getId()).getCreated()).isEqualTo(now);
assertThat(dataStore.get(oldExportRepo.getId())).isNull();
}
}

View File

@@ -53,7 +53,11 @@ import java.util.function.Supplier;
import static org.assertj.core.api.Assertions.assertThat;
import static org.mockito.ArgumentMatchers.any;
import static org.mockito.ArgumentMatchers.eq;
import static org.mockito.Mockito.*;
import static org.mockito.Mockito.anyString;
import static org.mockito.Mockito.mock;
import static org.mockito.Mockito.times;
import static org.mockito.Mockito.verify;
import static org.mockito.Mockito.when;
@ExtendWith(MockitoExtension.class)
class FullScmRepositoryExporterTest {
@@ -74,18 +78,21 @@ class FullScmRepositoryExporterTest {
private WorkdirProvider workdirProvider;
@Mock
private RepositoryExportingCheck repositoryExportingCheck;
@Mock
private RepositoryImportExportEncryption repositoryImportExportEncryption;
@InjectMocks
private FullScmRepositoryExporter exporter;
private Collection<Path> workDirsCreated = new ArrayList<>();
private final Collection<Path> workDirsCreated = new ArrayList<>();
@BeforeEach
void initRepoService() {
void initRepoService() throws IOException {
when(serviceFactory.create(REPOSITORY)).thenReturn(repositoryService);
when(environmentGenerator.generate()).thenReturn(new byte[0]);
when(metadataGenerator.generate(REPOSITORY)).thenReturn(new byte[0]);
when(repositoryExportingCheck.withExportingLock(any(), any())).thenAnswer(invocation -> invocation.getArgument(1, Supplier.class).get());
when(repositoryImportExportEncryption.optionallyEncrypt(any(), any())).thenAnswer(invocation -> invocation.getArgument(0));
}
@Test
@@ -95,7 +102,7 @@ class FullScmRepositoryExporterTest {
when(repositoryService.getRepository()).thenReturn(REPOSITORY);
when(workdirProvider.createNewWorkdir(anyString())).thenAnswer(invocation -> createWorkDir(temp, invocation.getArgument(0, String.class)));
ByteArrayOutputStream baos = new ByteArrayOutputStream();
exporter.export(REPOSITORY, baos);
exporter.export(REPOSITORY, baos, "");
verify(storeExporter, times(1)).export(eq(REPOSITORY), any(OutputStream.class));
verify(environmentGenerator, times(1)).generate();

View File

@@ -85,6 +85,8 @@ class FullScmRepositoryImporterTest {
@Mock
private UpdateEngine updateEngine;
@Mock
private RepositoryImportExportEncryption repositoryImportExportEncryption;
@Mock
private WorkdirProvider workdirProvider;
@InjectMocks
@@ -100,13 +102,20 @@ class FullScmRepositoryImporterTest {
@BeforeEach
void initTestObject() {
fullImporter = new FullScmRepositoryImporter(environmentCheckStep, metadataImportStep, storeImportStep, repositoryImportStep, repositoryManager);
fullImporter = new FullScmRepositoryImporter(
environmentCheckStep,
metadataImportStep,
storeImportStep,
repositoryImportStep,
repositoryManager,
repositoryImportExportEncryption);
}
@BeforeEach
void initRepositoryService() {
void initRepositoryService() throws IOException {
lenient().when(serviceFactory.create(REPOSITORY)).thenReturn(service);
lenient().when(service.getUnbundleCommand()).thenReturn(unbundleCommandBuilder);
lenient().when(repositoryImportExportEncryption.decrypt(any(), any())).thenAnswer(invocation -> invocation.getArgument(0));
}
@Test
@@ -116,7 +125,7 @@ class FullScmRepositoryImporterTest {
FileInputStream inputStream = new FileInputStream(emptyFile.toFile());
assertThrows(
ImportFailedException.class,
() -> fullImporter.importFromStream(REPOSITORY, inputStream)
() -> fullImporter.importFromStream(REPOSITORY, inputStream, "")
);
}
@@ -127,7 +136,7 @@ class FullScmRepositoryImporterTest {
InputStream importStream = Resources.getResource("sonia/scm/repository/import/scm-import.tar.gz").openStream();
assertThrows(
IncompatibleEnvironmentForImportException.class,
() -> fullImporter.importFromStream(REPOSITORY, importStream)
() -> fullImporter.importFromStream(REPOSITORY, importStream, "")
);
}
@@ -146,7 +155,7 @@ class FullScmRepositoryImporterTest {
void shouldImportScmRepositoryArchiveWithWorkDir() throws IOException {
InputStream stream = Resources.getResource("sonia/scm/repository/import/scm-import.tar.gz").openStream();
Repository repository = fullImporter.importFromStream(REPOSITORY, stream);
Repository repository = fullImporter.importFromStream(REPOSITORY, stream, "");
assertThat(repository).isEqualTo(REPOSITORY);
verify(storeImporter).importFromTarArchive(eq(REPOSITORY), any(InputStream.class));
@@ -161,7 +170,7 @@ class FullScmRepositoryImporterTest {
void shouldNotExistWorkDirAfterRepositoryImportIsFinished(@TempDir Path temp) throws IOException {
when(workdirProvider.createNewWorkdir(REPOSITORY.getId())).thenReturn(temp.toFile());
InputStream stream = Resources.getResource("sonia/scm/repository/import/scm-import.tar.gz").openStream();
fullImporter.importFromStream(REPOSITORY, stream);
fullImporter.importFromStream(REPOSITORY, stream, "");
boolean workDirExists = Files.exists(temp);
assertThat(workDirExists).isFalse();
@@ -171,7 +180,7 @@ class FullScmRepositoryImporterTest {
void shouldTriggerUpdateForImportedRepository() throws IOException {
InputStream stream = Resources.getResource("sonia/scm/repository/import/scm-import.tar.gz").openStream();
fullImporter.importFromStream(REPOSITORY, stream);
fullImporter.importFromStream(REPOSITORY, stream, "");
verify(updateEngine).update(REPOSITORY.getId());
}
@@ -179,7 +188,7 @@ class FullScmRepositoryImporterTest {
@Test
void shouldImportRepositoryDirectlyWithoutCopyInWorkDir() throws IOException {
InputStream stream = Resources.getResource("sonia/scm/repository/import/scm-import-stores-before-repository.tar.gz").openStream();
Repository repository = fullImporter.importFromStream(REPOSITORY, stream);
Repository repository = fullImporter.importFromStream(REPOSITORY, stream, "");
assertThat(repository).isEqualTo(REPOSITORY);
verify(storeImporter).importFromTarArchive(eq(REPOSITORY), any(InputStream.class));

View File

@@ -0,0 +1,111 @@
/*
* MIT License
*
* Copyright (c) 2020-present Cloudogu GmbH and Contributors
*
* Permission is hereby granted, free of charge, to any person obtaining a copy
* of this software and associated documentation files (the "Software"), to deal
* in the Software without restriction, including without limitation the rights
* to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
* copies of the Software, and to permit persons to whom the Software is
* furnished to do so, subject to the following conditions:
*
* The above copyright notice and this permission notice shall be included in all
* copies or substantial portions of the Software.
*
* THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
* IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
* FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
* AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
* LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
* OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
* SOFTWARE.
*/
package sonia.scm.importexport;
import com.google.common.io.ByteSource;
import org.junit.jupiter.api.Test;
import java.io.ByteArrayInputStream;
import java.io.ByteArrayOutputStream;
import java.io.IOException;
import java.io.InputStream;
import java.io.OutputStream;
import java.nio.charset.StandardCharsets;
import static org.assertj.core.api.Assertions.assertThat;
import static org.junit.jupiter.api.Assertions.assertThrows;
class RepositoryImportExportEncryptionTest {
private final RepositoryImportExportEncryption encryption = new RepositoryImportExportEncryption();
@Test
void shouldNotEncryptWithoutPassword() throws IOException {
String content = "my content";
String secret = "";
ByteArrayOutputStream baos = new ByteArrayOutputStream();
OutputStream os = encryption.optionallyEncrypt(baos, secret);
os.write(content.getBytes());
os.flush();
assertThat(os).hasToString(content);
}
@Test
void shouldNotDecryptWithoutPassword() throws IOException {
String content = "my content";
String secret = "";
ByteArrayInputStream bais = new ByteArrayInputStream(content.getBytes());
InputStream is = encryption.optionallyDecrypt(bais, secret);
ByteSource byteSource = new ByteSource() {
@Override
public InputStream openStream() {
return is;
}
};
String result = byteSource.asCharSource(StandardCharsets.UTF_8).read();
assertThat(result).isEqualTo(content);
}
@Test
void shouldEncryptAndDecryptContentWithPassword() throws IOException {
String content = "my content";
String secret = "secretPassword";
ByteArrayOutputStream baos = new ByteArrayOutputStream();
OutputStream os = encryption.optionallyEncrypt(baos, secret);
os.write(content.getBytes());
os.flush();
os.close();
assertThat(baos.toString()).isNotEqualTo(content);
InputStream is = encryption.optionallyDecrypt(new ByteArrayInputStream(baos.toByteArray()), secret);
ByteSource byteSource = new ByteSource() {
@Override
public InputStream openStream() {
return is;
}
};
String result = byteSource.asCharSource(StandardCharsets.UTF_8).read();
assertThat(result).isEqualTo(content);
}
@Test
void shouldFailOnDecryptIfNotEncrypted() {
String content = "my content";
String secret = "secretPassword";
ByteArrayInputStream notEncryptedStream = new ByteArrayInputStream(content.getBytes());
assertThrows(IOException.class, () -> encryption.optionallyDecrypt(notEncryptedStream, secret));
}
}

View File

@@ -39,6 +39,7 @@ import sonia.scm.SCMContext;
import sonia.scm.config.ScmConfiguration;
import sonia.scm.group.Group;
import sonia.scm.group.GroupManager;
import sonia.scm.importexport.ExportService;
import sonia.scm.security.AnonymousMode;
import sonia.scm.security.PermissionAssigner;
import sonia.scm.security.PermissionDescriptor;
@@ -82,6 +83,9 @@ class SetupContextListenerTest {
@Mock
private GroupManager groupManager;
@Mock
private ExportService exportService;
@Mock
private PermissionAssigner permissionAssigner;
@@ -209,6 +213,13 @@ class SetupContextListenerTest {
verify(groupManager, never()).create(any());
}
@Test
void shouldCleanupUnfinishedRepositoryExports() {
setupContextListener.contextInitialized(null);
verify(exportService).cleanupUnfinishedExports();
}
private void verifyAdminPermissionsAssigned() {
ArgumentCaptor<String> usernameCaptor = ArgumentCaptor.forClass(String.class);
ArgumentCaptor<Collection<PermissionDescriptor>> permissionCaptor = ArgumentCaptor.forClass(Collection.class);