Before this change the search uses a single index which distinguishes types (repositories, users, etc.) with a field (_type).
But it has turned out that this could lead to problems, in particular if different types have the same field and uses different analyzers for those fields. The following links show even more problems of a combined index:
https://www.elastic.co/blog/index-vs-typehttps://www.elastic.co/guide/en/elasticsearch/reference/6.0/removal-of-types.html
With this change every type becomes its own index and the SearchEngine gets an api to modify multiple indices at once to remove all documents from all indices, which are related to a specific repository, for example.
The search uses another new api to coordinate the indexing, the central work queue.
The central work queue is able to coordinate long-running or resource intensive tasks. It is able to run tasks in parallel, but can also run tasks which targets the same resources in sequence. The queue is also persistent and can restore queued tasks after restart.
Co-authored-by: Konstantin Schaper <konstantin.schaper@cloudogu.com>
Add option to encrypt repository exports with a password and add possibility to decrypt them on repository import. Also make the repository export asynchronous. This implies that the repository export will be created on the server and can be downloaded multiple times. The repository export will be deleted automatically 10 days after creation.
The default (XML) store of SCM-Manager does not distinguish between config and config entry stores in regards to
storage locations. Nonetheless, we want to make a difference in export files, so that other store providers can handle
these stores differently. To do so, this change adds an attribute to the top level xml element of config entry stores
to mark them. In exports, these store files can now be exported in a different folder. To mark existing stores, this
introduces an update step.
* Add store exporter to collect the repository metadata
* Add EnvironmentInformationXmlGenerator
* Collect export data and put into compressed tar archive output stream
* Create full repository export endpoint.
* Add full repository export to ui
* Ignore irrelevant files from config store directory
* write metadata stores to file since a baos could teardown the server memory
* Migrate store name for git lfs files (#1504)
Changes the directory name for the git LFS blob store by
removing the repository id from the store name.
This is necessary for im- and exports of lfs blob stores,
because the original name had the repository id as a part
of it and therefore the old store would not be found when
the repository is imported with another id.
Existing blob files will be moved to the new store location
by an update step.
Co-authored-by: Eduard Heimbuch <eduard.heimbuch@cloudogu.com>
* Introduce util for migrations (#1505)
With this util it is more simple to rename
or delete stores.
* Rename files in export
Co-authored-by: René Pfeuffer <rene.pfeuffer@cloudogu.com>
This adds a flag "archived" to repositories. Repositories marked with this can no longer be modified in any way. To do this, we switch to a new version of Shiro Static Permissions (sdorra/shiro-static-permissions#4) and specify a permission guard to check for every permission request, whether the repository in question is archived or not. Further we implement checks in stores and other activies so that no writing request may be executed by mistake.
Co-authored-by: Eduard Heimbuch <eduard.heimbuch@cloudogu.com>
