Nemcio/SCM-Manager
1
0
Fork 0
mirror of https://github.com/scm-manager/scm-manager.git synced 2025-11-15 17:56:17 +01:00
Code Issues Packages Projects Releases Activity

One index per type and parallel indexing (#1781)

Browse Source 
Before this change the search uses a single index which distinguishes types (repositories, users, etc.) with a field (_type).
But it has turned out that this could lead to problems, in particular if different types have the same field and uses different analyzers for those fields. The following links show even more problems of a combined index:

    https://www.elastic.co/blog/index-vs-type
    https://www.elastic.co/guide/en/elasticsearch/reference/6.0/removal-of-types.html

With this change every type becomes its own index and the SearchEngine gets an api to modify multiple indices at once to remove all documents from all indices, which are related to a specific repository, for example.

The search uses another new api to coordinate the indexing, the central work queue.
The central work queue is able to coordinate long-running or resource intensive tasks. It is able to run tasks in parallel, but can also run tasks which targets the same resources in sequence. The queue is also persistent and can restore queued tasks after restart.

Co-authored-by: Konstantin Schaper <konstantin.schaper@cloudogu.com>
This commit is contained in:
Sebastian Sdorra
2021-08-25 15:40:11 +02:00
committed by GitHub
parent 44f25d6b15
commit 0a26741ebd
72 changed files with 4536 additions and 1420 deletions
Download Patch File Download Diff File Expand all files Collapse all files

22
scm-dao-xml/src/main/java/sonia/scm/store/StoreConstants.java
View File

@@ -21,12 +21,12 @@
* OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
* SOFTWARE.
*/
package sonia.scm.store;
/**
* Store constants for xml implementations.
*
*
* @author Sebastian Sdorra
*/
public class StoreConstants
@@ -36,6 +36,24 @@ public class StoreConstants
public static final String CONFIG_DIRECTORY_NAME = "config";
/**
* Name of the parent of data or blob directories.
* @since 2.23.0
*/
public static final String VARIABLE_DATA_DIRECTORY_NAME = "var";
/**
* Name of data directories.
* @since 2.23.0
*/
public static final String DATA_DIRECTORY_NAME = "data";
/**
* Name of blob directories.
* @since 2.23.0
*/
public static final String BLOG_DIRECTORY_NAME = "data";
public static final String REPOSITORY_METADATA = "metadata";
public static final String FILE_EXTENSION = ".xml";