GitSearch Application

This document presents the basic idea of the GitLab search service and the setup of the development environment.

Concept

Ideas

The main idea of the frontend of the CodeAbility Sharing Platform is providing a search engine which allows searching the projects provided in any repository that lies within the group sharing. Additionally other repositories are indexed, if they contain a metadata-file in the root directory of the repository. The search engine should also allow users to search/filter for specific metadata (e.g., license(s), programming language(s)). Search results should be presented visually appealing so that users can easily find and use resources without any prior knowledge (especially Git). Moreover, the Sharing Platform should provide a starting and an imprint page, as well as data privacy pages.

Technology

The application was generated with JHipster v7.9.3. JWT-tokens are used for authentication. In production MySQL is utilized as database management system. H2 with disk-based persistence is employed in development. Maven is used for building the backend.

Production/Development environment

Before starting the search, please ensure that the backend is up and running! Afterward, the following steps can be executed to start the search.

The ci/cd pipeline is configured to automatically build and deploy the application to the development (CI task ‘deploy’) or the production environment (CI task ‘deploy-prod’).

The required environment variables are defined in the GitLab project settings (CI/CD -> Variables) and .env file.

Local development environment

A local development environment may help develop, test, and debug the application.

It is however quite resource intensive and requires a lot of setup. Therefore the recommended way to develop the GitLab search is to use docker infrastructure on the development environment.

I.e. use the gitlab instance at https://sharing.codeability-austria.uibk.ac.at/dashboard. Additionally to use the elasticsearch index, open an ssh-tunnel to the elasticsearch docker container.

ssh -L 9200:sharing_elasticsearch:9200 <username>@codeability-austria.uibk.ac.at

or in windows (e.g. with putty)

Frontend

Clone the GitLab search project
Start the spring backend of GitLab search in your IDE or by running ./mvnw. If this does not work, running ./mvnw clean && ./mvnw might fix it.
To enable automatic recompilation upon changes in the frontend code (Angular), run npm start in the root directory of the repository.

After the frontend and backend were started successfully, the services should be available on the following default ports.

Service	Port
GitLab search (no auto-update)	8080
GitLab search (auto-update)	9000
Elasticsearch (if enabled)	9200
GitLab (optional)	10082

If port 9200 is accessible, it can be useful to check http://localhost:9200/metadata/_search for debugging purposes. When data has been indexed by Elasticsearch it should show up there in JSON format. Note that GitLab will only tell Elasticsearch to index repositories in the group sharing. Only valid metadata is indexed.

Authentication and Integration with GitLab

It is important that the user of the search platform does not need to register / login twice for the CodeAbility repository

To this end a keycloak server is installed.

Basic Concepts

GitLab offers an OAuth2 Service to share authentication and further to allow access to other resources. In the following we describe the configuration issues for GitLab.

Establish Preconditions

gitsearch must run behind an (apache) reverse proxy that forwards the original URL. In order to do so, add

SSLProxyEngine    On

ProxyPreserveHost On
ProxyRequests Off
RequestHeader set X-Forwarded-Proto "https"
ProxyPass / http://codeability-austria.uibk.ac.at:10084/
ProxyPassReverse / http://codeability-austria.uibk.ac.at:10084/

Configuration in GitLab

As an administrator choose “Admin Area” -> “Applications” and select “New Application” or edit for an existing Application. In the form enter

any descriptive name
as redirect URI the full URI of the gitsearch endpoint (e.g. https://dev-exchange.codeability-austria.uibk.ac.at/login/oauth2/code/oidc). You can specify serveral urls.
select “Trusted” (to avoid nasty confirmation questions from gitlab)
deselect “Confidential” (not necessary)
select api, openid and read_user

After submission gitlab will have generated

an application ID and
an application secret

Configuration in gitsearch

Enter the ids from the last section in application{-prod | -dev | -staging}.properties under

security:
  oauth2:
    client:
      provider:
        oidc:
          issuer-uri: https://sharing.codeability-austria.uibk.ac.at
      registration:
        oidc:
          client-id: 149276ac11138d9ba72fb3cd12815e3fa2f372866df0eac0f7d1aae5fdffea24
          client-secret: 6f480635241f420a361581f4837594ea6f48f5ee6f515c1aa89f325dd922dbb0

The issuer-uri must be changed to the base url of your gitlab installation (e.g. https://sharing.codeability.uibk.ac.at)

For the “Entwicklungsserver” an additional staging profile was defined, in order to override certain production configuration settings. To activate it, edit the docker-compose file src/main/docker/gitsearch.yml

gitsearch-app:
  image: docker.uibk.ac.at:443/csar9407/gitsearch:development-8ad4fb188a8358c34355cebc0400ae731cdf966e
  #     image: docker.uibk.ac.at:443/csar9407/gitsearch:master-1bf1f21ceee62f335eeb4b070d93443cdbf4a14b
  container_name: sharing_search
  environment:
    - _JAVA_OPTIONS=-Xmx512m -Xms256m
    - SPRING_PROFILES_ACTIVE=prod,swagger,staging

Note

Remark, to register several oauth2 service providers, add them under different “oidc” tags.