【AWS】s3cmd 使用

Posted by 西维蜀黍 on 2020-05-05, Last Modified on 2021-10-17

Install

Windows

直接执行.py源文件

Install Python

Download and install python 3

https://www.python.org/downloads/

After installing python, append the Python in Path environment variable. Open command prompt and verify python version.

C:> python3 --version
Python 3.9.4

Download s3cmd

$ git clone https://github.com/s3tools/s3cmd.git

# 解压

$ cd s3cmd

$ python3 s3cmd --configure

Verify

$ python3 s3cmd ls

2014-02-03 06:37  s3://tecadmin
2014-03-29 07:56  s3://tecadminbackups

使用 pip 安装

$ pip3 install s3cmd

# 把pip3的全局脚本设置到PATH环境变量中,在我的case下,是C:\Users\swsmi\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.9_qbz5n2kfra8p0\LocalCache\local-packages\Python39\Scripts

Ubuntu

$  pip3 install s3cmd

Config

$ s3cmd --configure

Enter new values or accept defaults in brackets with Enter.
Refer to user manual for detailed description of all options.

Access key and Secret key are your identifiers for Amazon S3. Leave them empty for using the env variables.
Access Key: AKIAWRUQZUW2PD3P3JNJ
Secret Key: rtPxCJMxShBtZNyOomomTepHpM6YgCXdcCgOu3Dd
Default Region [US]:

Use "s3.amazonaws.com" for S3 Endpoint and not modify it to the target Amazon S3.
S3 Endpoint [s3.amazonaws.com]:

Use "%(bucket)s.s3.amazonaws.com" to the target Amazon S3. "%(bucket)s" and "%(location)s" vars can be used
if the target S3 system supports dns based buckets.
DNS-style bucket+hostname:port template for accessing a bucket [%(bucket)s.s3.amazonaws.com]:

Encryption password is used to protect your files from reading
by unauthorized persons while in transfer to S3
Encryption password:
Path to GPG program [/usr/bin/gpg]:

When using secure HTTPS protocol all communication with Amazon S3
servers is protected from 3rd party eavesdropping. This method is
slower than plain HTTP, and can only be proxied with Python 2.7 or newer
Use HTTPS protocol [Yes]: Yes

On some networks all internet access must go through a HTTP proxy.
Try setting it here if you can't connect to S3 directly
HTTP Proxy server name:

New settings:
  Access Key: AKIAWRUQZUW2PD3P3JNJ
  Secret Key: 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
  Default Region: US
  S3 Endpoint: s3.amazonaws.com
  DNS-style bucket+hostname:port template for accessing a bucket: %(bucket)s.s3.amazonaws.com
  Encryption password:
  Path to GPG program: /usr/bin/gpg
  Use HTTPS protocol: True
  HTTP Proxy server name:
  HTTP Proxy server port: 0

Test access with supplied credentials? [Y/n] Y
Please wait, attempting to list all buckets...
Success. Your access key and secret key worked fine :-)

Now verifying that encryption works...
Not configured. Never mind.

Save settings? [y/N]
Configuration saved to '/home/swubuntu/.s3cfg'

测试一下

$ s3cmd ls

2014-02-03 06:37  s3://tecadmin
2014-03-29 07:56  s3://tecadminbackups

Usage

同步文件到 S3

# 两者没有区别
$ s3cmd sync --recursive "/Working/bin" s3://swtestiznzwbzxihzi --delete-removed
$ s3cmd sync --recursive "/Working/bin" s3://swtestiznzwbzxihzi/ --delete-removed

# 这会把/Working/下的bin/文件夹放到s3://swtestiznzwbzxihzi/的根目录下,比如有/Working/bin/1,s3://swtestiznzwbzxihzi/bin/1 中就是对应的 1文件
$ s3cmd sync --recursive "/Working/bin" s3://swtestiznzwbzxihzi/ --delete-removed

# 这会把/Working/bin/下的所有文件都放到s3://swtestiznzwbzxihzi/的根目录下,比如有/Working/bin/1,s3://swtestiznzwbzxihzi/中就会包含1文件
$ s3cmd sync --recursive "/Working/bin/" s3://swtestiznzwbzxihzi/ --delete-removed

Reference