Clone All Projects From Gitlab

1 minute read

At my company, we haven’t applied the mono repository yet, so that I have to clone all the repos manually. The problem is there are hundreds of them, and I’m too lazy to go one by one copy the link then paste it to the terminal to clone. I came up with a solution: write a script to clone them all.

1. Get the repo list

Gitlab using the server rendering for the project page: https://gitlab.com/dashboard/projects, so that there is no API to inspect to get that list. Hmm, modern problems required modern solutions. I went through each page, write a script to console to get the list of repo name.

Array.from( document.getElementsByClassName("project-name") ).map(e => e.textContent)

This step, we still need to manually copy it to a text file (name repos.text, for example) to use in the next step.

2. Run script to clone them all

Yeah, you no need to write, the script is here:

const fs = require("fs");
const exec = require("child_process").exec;

const filename = "./repos.text";
const tasks = [];

const repos = fs
  .readFileSync(filename)
  .toString()
  .split("\n");

// Remove last \n which splited an empty string after \n
repos.pop();

repos.forEach(repo => {
  tasks.push(
    new Promise(resolve => {
      const command = `git clone git@gitlab.com:your-company/${repo}.git`;
      console.log(command);

      exec(command, (error, stdout, stderr) => {
        if (error) {
          console.error(`exec error: ${error}`);
          resolve();
          return;
        }
        console.log(`stderr: ${stderr}`);
        resolve();
      });
    })
  );
});

Promise.all(tasks).then(() => {
  console.log("Done");
});

The idea is quite simple, getting the repo list then exec several clone command.

I’m satisfied with the script. Now I wait for a few minutes then all my repo there, it took time at first but right now anyone read this, you will save a lot of time.

Cheers.

Updated:

Comments