git Bitbucket 克隆所有团队存储库
声明：本页面是StackOverFlow热门问题的中英对照翻译，遵循CC BY-SA 4.0协议，如果您需要使用它，必须同样遵循CC BY-SA许可，注明原文地址和作者信息，同时你必须将它归于原作者（不是我）：StackOverFlow
原文地址: http://stackoverflow.com/questions/40429610/
Warning: these are provided under cc-by-sa 4.0 license. You are free to use/share it, But you must attribute it to the original authors (not me):
StackOverFlow
Bitbucket clone all team repositories
提问by Imran
I'd like to clone all my bitbucket team repositories using a bash script and http. I have found a few examples that use the Bitbucket api however, they all seem to return no repositories. Any ideas? Using mac.
我想使用 bash 脚本和 http 克隆我所有的 bitbucket 团队存储库。我发现了一些使用 Bitbucket api 的示例，但是，它们似乎都没有返回存储库。有任何想法吗？使用 mac。
回答by Eric Nord
Ensure you have your ssh keys setup in bitbucket and git is installed by cloning one repo manually:
确保您在 bitbucket 中设置了 ssh 密钥，并通过手动克隆一个 repo 来安装 git：
This will clone all repos that are owned by you:
这将克隆您拥有的所有存储库：
USER=bitbucket_username; curl --user ${USER} https://api.bitbucket.org/2.0/repositories/${USER} | grep -o '"ssh:[^ ,]\+' | xargs -L1 git clone
To backup your team repositories use the same script but hard code your bitbucket team name like this:
要备份您的团队存储库，请使用相同的脚本，但对您的 bitbucket 团队名称进行硬编码，如下所示：
USER=bitbucket_username; curl --user ${USER} https://api.bitbucket.org/2.0/repositories/TEAMNAME | grep -o '"ssh:[^ ,]\+' | xargs -L1 git clone
Here's a better way:
这是一个更好的方法：
curl -u https://api.bitbucket.org/1.0/users/TEAMNAME > repoinfo
for repo_name in `cat repoinfo | sed -r 's/("name": )/\n/g' | sed -r 's/"name": "(.*)"//' | sed -e 's/{//' | cut -f1 -d\" | tr '\n' ' '`
do
echo "Cloning " $repo_name
git clone [email protected]:TEAMNAME/$repo_name.git
echo "---"
done
回答by Nikolay Galkin
Here is my simple solution. Make file downloader.sh
这是我的简单解决方案。制作文件downloader.sh
#!/bin/bash
USER=
TEAM=
rm -rf "$TEAM" && mkdir "$TEAM" && cd $TEAM
NEXT_URL="https://api.bitbucket.org/2.0/repositories/${TEAM}?pagelen=100"
while [ ! -z $NEXT_URL ]
do
curl -u $USER $NEXT_URL > repoinfo.json
jq -r '.values[] | .links.clone[1].href' repoinfo.json > ../repos.txt
NEXT_URL=`jq -r '.next' repoinfo.json`
for repo in `cat ../repos.txt`
do
echo "Cloning" $repo
if echo "$repo" | grep -q ".git"; then
command="git"
else
command="hg"
fi
$command clone $repo
done
done
cd ..
you can run it by:
您可以通过以下方式运行它：
sh downloader.sh username teamname # or username instead team name
回答by Eduardo Folly
Another alternative using jq.
使用jq 的另一种选择。
#!/bin/bash
user=username:password
curl -u $user 'https://api.bitbucket.org/2.0/user/permissions/teams?pagelen=100' > teams.json
jq -r '.values[] | .team.username' teams.json > teams.txt
for team in `cat teams.txt`
do
echo $team
rm -rf "${team}"
mkdir "${team}"
cd "${team}"
url="https://api.bitbucket.org/2.0/repositories/${team}?pagelen=100"
echo $url
curl -u $user $url > repoinfo.json
jq -r '.values[] | .links.clone[0].href' repoinfo.json > repos.txt
for repo in `cat repos.txt`
do
echo "Cloning" $repo
git clone $repo
done
cd ..
done
回答by simpleuser
You can only use a simple command if you have less than 100 repositories and add 'pagelen=100' to the query, since that is the most that the bitbucket API will report at one time. If you have more than 100 repositories, you need to process the "next" link in the JSON returned, to get the URL to query the next set of repositories, which would be easier with a script.
如果您的存储库少于 100 个并在查询中添加 'pagelen=100'，则只能使用简单的命令，因为这是 bitbucket API 一次报告的最多次数。如果你有超过 100 个仓库，你需要处理返回的 JSON 中的“next”链接，以获取查询下一组仓库的 URL，这会更容易使用脚本。
If you use http to clone instead of ssh, then you either need to enter the password for protected repositories, or else get a bitbucket app password and modify the URLs to insert that into them, so they look like:
如果您使用 http 而不是 ssh 进行克隆，那么您需要输入受保护存储库的密码，或者获取 bitbucket 应用程序密码并修改 URL 以将其插入其中，因此它们看起来像：
https://bitbucketuserhere:[email protected]/teamorusername/repositoryname.git
Additionally, cloning will not get all versions of git LFS files, so be aware of that. According to bitbucket, use 'git lfs fetch --all' to copy all LFS file versions locally.
此外，克隆不会获得所有版本的 git LFS 文件，因此请注意这一点。根据 bitbucket，使用 'git lfs fetch --all' 在本地复制所有 LFS 文件版本。
To get the list of your personal repositories, use a URL like:
要获取您的个人存储库列表，请使用如下 URL：
https://api.bitbucket.org/2.0/repositories/BITBUCKETUSERNAME?pagelen=100
https://api.bitbucket.org/2.0/repositories/BITBUCKETUSERNAME?pagelen=100
To get the list of your team repositories, use a URL like this to get the list of all repositories you are a member of:
要获取您的团队存储库列表，请使用这样的 URL 来获取您所属的所有存储库的列表：
https://api.bitbucket.org/2.0/repositories/TEAMNAME?pagelen=100&role=member
https://api.bitbucket.org/2.0/repositories/TEAMNAME?pagelen=100&role=member
The following is an example perl script you could use to clone and then maintain copies of your repositories, using http instead of ssh to fetch. It makes --mirror clones instead of a fully populated working copy (perfect for moving or disaster recovery). It does notback up all LFS files.
下面是一个示例 perl 脚本，您可以使用它来克隆和维护存储库的副本，使用 http 而不是 ssh 来获取。它使 --mirror 克隆而不是完全填充的工作副本（非常适合移动或灾难恢复）。它并没有备份所有文件LFS。
#!/usr/bin/env perl
use warnings;
use strict;
use JSON::Parse 'parse_json';
# CONFIGURATION:
# Bitbucket team or user name to get list of repositories from
my $teamORuserName = "myteam";
# Bitbucket app password with access to query the API for the
# list of repositories. Format: "user-name:app-token"
my $appPassword= "frank-james:LAYDxtc8H6FGKUZeHEef";
#------------------------------------------------------------------------------
my $nextPageLink = "https://api.bitbucket.org/2.0/repositories/$teamORuserName?pagelen=100&role=member";
while (defined $nextPageLink)
{
$nextPageLink =~ m/page=(\d+)/;
print "Fetching page " . ( || 1). "\n";
my $response = `curl -q --silent --request GET --user '$appPassword' '$nextPageLink'`;
my $json = parse_json($response);
my $values = $json->{values};
foreach my $repo (@$values)
{
die "'$repo->{name}' is not a 'git' repo: $repo->{scm}" unless $repo->{scm} eq "git";
my $links = $repo->{links} || die "no links data for '$repo->{name}'";
my $clones = $links->{clone} || die "no clone data for '$repo->{name}'";
my $url = $clones->[0]->{href} || die "no clone url found for $repo->{name}";
# use uuid as directory name, to survive project name changes
my $uuid = $repo->{uuid}; $uuid =~ s/[\{\}]//g;
if (not -e $uuid)
{
print "cloning '$repo->{name}' into $uuid\n";
# replace user name with token to avoid password prompts
$url =~ s|(https?://).+(\@bitbucket.org)|$appPassword|;
system("git clone --progress --mirror '$url' $uuid") == 0 or die "clone failed";
# make a human friendly link to current repository name
symlink $uuid, $repo->{slug} or warn "symlink failed: $!";
}
else
{
print "updating '$repo->{name}' in $uuid\n";
system("cd $uuid && git fetch --all --tags --prune") == 0 or die "fetch failed";
}
print "\n";
}
$nextPageLink = $json->{next};
}
exit 0;
回答by Ramkumar Natarajan
Here is a python script for cloning all the team or user's repositories in bitbucket. As the team repositories are usually private, I took care of that while using the bitbucket API. So just enter your bitbucket username, password, and the team's username and it will take care of cloning all the team repository for you.
这是一个 python 脚本，用于在 bitbucket 中克隆所有团队或用户的存储库。由于团队存储库通常是私有的，因此我在使用 bitbucket API 时会处理这些问题。因此，只需输入您的 bitbucket 用户名、密码和团队用户名，它就会为您克隆所有团队存储库。
import subprocess
import json
cmd = "curl -u <bitbucket_username>:<bitbucket_password> https://api.bitbucket.org/2.0/repositories/<team_name_or_project_name>"
cmd = cmd.split()
while 1:
from_api = subprocess.Popen(cmd, shell=False, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
from_api = from_api.communicate()
json_from_api = json.loads(from_api[0])
for unit_dict in json_from_api["values"]:
clone_cmd = "git clone " + unit_dict["links"]["clone"][1]["href"]
clone_cmd = clone_cmd.split()
clone_out = subprocess.call(clone_cmd, shell=False)
if "next" not in json_from_api:
break
else:
cmd[-1] = json_from_api["next"]
回答by Gladwin Henald
Here is a node script to download all repositories in a bitbucket account. Please don't forget to add necessary npm packages.
这是一个节点脚本，用于下载 bitbucket 帐户中的所有存储库。请不要忘记添加必要的 npm 包。
const argv = require('yargs').argv
const request = require('request');
const nodegit = require('nodegit');
let repos = [];
let opts = {
fetchOpts: {
callbacks: {
credentials: function() {
return nodegit.Cred.userpassPlaintextNew(argv.username, argv.password);
},
certificateCheck: function() {
return 1;
}
}
}
};
function cloneRepository(index) {
let repo = repos[index];
console.log('Cloning ' + repo.full_name);
nodegit.Clone(repo.links.clone[0].href, 'repositories/' + repo.full_name, opts)
.then(function(repo) {
if (repos.length - 1 == index) {
console.log("All repositories cloned");
} else {
cloneRepository(index + 1);
}
})
.catch(function(err) {
if (err) {
console.log(err);
}
});
}
function loadRepositories(url) {
request.get(url, {
'auth': {
'user': argv.username,
'pass': argv.password
}
}, function (err, response, body) {
if (err) return console.log(err);
let data = JSON.parse(body);
for (var i = 0; i < data.values.length; i++) {
repos.push(data.values[i]);
}
if (data.next){
loadRepositories(data.next);
} else if (repos.length > 0) {
console.log('Started cloning..');
cloneRepository(0);
} else {
console.log("No repositories found");
}
});
}
if (argv.username && argv.password) {
console.log('Loading all repositories..');
loadRepositories('https://api.bitbucket.org/2.0/repositories/?role=member');
} else {
console.log('Please specify both the --username and --password options');
}
You can also check out this GitHub repository. Bitbucket Repository Downloader
您还可以查看此 GitHub 存储库。Bitbucket 存储库下载器
回答by Geotti
Building upon @eric-nord's answer, I've replaced sed with jqto get to the repo slug (which I found simpler, considering bitbucket outputs json).
基于@eric-nord 的回答，我已经用jq替换了 sed以获得 repo slug（我发现它更简单，考虑到 bitbucket 输出 json）。
So, if you have fewer than 100 repositories and you're looking to clone the repositories for a specific project, the following shell script should do the trick.
因此，如果您的存储库少于 100 个，并且您希望为特定项目克隆存储库，则以下 shell 脚本应该可以解决问题。
#!/usr/bin/env sh
if [ $# -eq 0 ] || [ "" == "-?" ] || [ "" == "--help" ] || [ "" == "-h" ]
then
echo "Usage: `basename import subprocess
import json
import cookielib
import urllib2
import base64
def _create_opener(proxy=None):
cj = cookielib.LWPCookieJar()
cookie_handler = urllib2.HTTPCookieProcessor(cj)
if proxy:
proxy_handler = urllib2.ProxyHandler(proxy)
opener = urllib2.build_opener(cookie_handler, proxy_handler)
else:
opener = urllib2.build_opener(cookie_handler)
return opener
def get_repos(_opener, _auth, user, password, team_username, paging):
query_url = 'https://bitbucket.org/!api/2.0/repositories/'+team_username+paging
try:
req = urllib2.Request(query_url, None, {"Authorization": _auth })
handler = _opener.open(req)
except urllib2.HTTPError, e:
print e.headers
raise e
for unit_dict in json.load(handler)["values"]:
clone_cmd = "git clone " + unit_dict["links"]["clone"][0]["href"].replace('https://'+user,'https://'+password+':'+password)
clone_cmd = clone_cmd.split()
clone_out = subprocess.call(clone_cmd, shell=False)
encodedstring = base64.encodestring("%s:%s" % (user, password))[:-1]
_auth = "Basic %s" % encodedstring
_opener = _create_opener()
get_repos(_opener,_auth,'bitbucket-user','bitbucket-password','team-username','pagelen=100&page=1')
` <USERNAME> <PROJECTNAME>"
echo "e.g. `basename $> brew install jensim/bitbucket_server_cli/bitbucket_server_cli
` some_user some_project"
exit 1
fi
curl -u https://your.bitbucket.url/rest/api/1.0/projects//repos?limit=100 > repos.json
for repo_name in `cat repos.json | jq -r '.values[] .slug'`
do
echo "cloning" $repo_name
git clone https://@your.bitbucket.url/scm//$repo_name.git .//$repo_name
done
回答by Milix
This is what worked for me in Python:
这就是在 Python 中对我有用的方法：
$> bitbucket_server_cli clone
BitBucket server address: http://localhost
BitBucket username: jensim
? BitBucket password · ********
Clone/update all projects yes
Fetching users [00:00:15] [########################################] 2011/2011 (eta:0s)
Fetching projects [00:00:00] [########################################] 35/35 (eta:0s)
Working repos [00:01:07] [########################################] 1337/1337 (eta:0s)
回答by Jens
I built a CLI in rust for concurrent cloning and pulling git repositories (user and project). https://github.com/jensim/bitbucket_server_cliIt works with interactive or batch-mode, has shell completion and ability to remember choices in the interactive mode, for a faster user experience.
我用 Rust 构建了一个 CLI，用于并发克隆和拉取 git 存储库（用户和项目）。 https://github.com/jensim/bitbucket_server_cli它在交互或批处理模式下工作，具有 shell 完成和在交互模式下记住选择的能力，以获得更快的用户体验。
For a mac user like yourself :
对于像你这样的 mac 用户：
#!/bin/bash
USER='yourBitBucketUsername' #not email
PASS='yourPassword'
TEAM='teamNameInBitbucket'
curl -u $USER:$PASS https://api.bitbucket.org/1.0/users/$TEAM > repoinfo
for repo_name in `cat repoinfo | sed -r 's/("name": )/\n/g' | sed -r 's/"name": "(.*)"//' | sed -e 's/{//' | cut -f1 -d\" | tr '\n' ' '`
do
echo "Cloning " $repo_name
git clone https://[email protected]/$TEAM/$repo_name.git
echo "---"
done
Disclaimer : This is quite experimental, so its mostly used by mac people at the moment, as far as I'm aware.. But there are windows and debian binaries available on Github Releases. Also, this works concurrently, with a configurable number of simultanious http requests/git clone, in order not to kill your CI pipelines
免责声明：这是相当实验性的，所以它目前主要由 mac 人使用，据我所知..但是 Github Releases 上有 Windows 和 debian 二进制文件。此外，这同时工作，具有可配置数量的同时 http 请求/git clone，以免杀死您的 CI 管道
回答by Marko
Here's a Bash script that downloads via https.
这是一个通过https.
Save to a file download_bitbucket_repos.sh, chmod +x download_bitbucket_repos.shthen ./download_bitbucket_repos.sh.
保存到文件download_bitbucket_repos.sh，chmod +x download_bitbucket_repos.sh然后./download_bitbucket_repos.sh.
