Object Storage

On this page, you can find an explanation of how to use the Cloud Console Object Storage with the Python API or with the client S3 Browser

Table of contents

  1. Introduction
  2. Object Storage page
  3. Create Object Storage credentials
  4. Show secret key of the Object Storage credentials
  5. Re-generate secret key of the Object Storage credentials
  6. Delete the Object Storage credentials
  7. Use the Object Storage with the Python API
    1. Creating connection
    2. Listing owned buckets
    3. Creating a bucket
    4. Creating an object
    5. Listing a bucket's content
    6. Changing an object's ACL (access control lists)
    7. Generating object download URL
    8. Deleting an object
    9. Deleting a bucket
  8. Use the Object Storage with the client S3 Browser

Introduction

The Cloud Console Object Storage makes it possible to store data simply and cost-effectively. It provides a fully distributed, API-accessible storage platform that can be integrated directly into applications or used for backup, archiving, and data retention.

It’s built for scale and optimized for durability, availability, and concurrency across the entire data set. 

An Object Storage API differs from a conventional filesystem: instead of directories and files, you manipulate containers where you store objects. A container can hold millions of objects.

There is no notion of hierarchy with containers: you cannot nest a container within another, however, you can emulate a nested folder structure with a naming convention for your objects.

Before we start to configure Object Storage we need to create credentials that include the access key and secret key. 

Object Storage page

To get to the Object Storage page on the Homepage you need to select the Storage from the VIRTUAL DATACENTER block and click the Object Storage TAB:

st1

st2

This action will redirect you to the Object Storage page where you can find all created Object Storage's credentials in the selected Project with information about them HeadersCreate buttonSearch bar and Actions icon which opens a list of available management actions for the selected Object Storage:

st4

Object Storage headers include:

  • Access key - shows the access key of the Object Storage credentials;
  • Endpoint - shows the endpoint of the Object Storage credentials.
Actions icon opens the next list of available management actions:
  • Show access secret - by selecting this option you can find the secret key of the created Object Storage credentials;

  • Re-generate secret -  by selecting this option you can re-generate the secret key of the created Object Storage credentials;

  • Delete storage - this option is for Object Storage credentials deletion.

So, from this page you can:

  • review all Object Storage credentials created in the selected Project;
  • create more Object Storage credentials;
  • find the secret key of the created Object Storage credentials;
  • re-generate the secret key of the created Object Storage credentials
  • delete unnecessary Object Storage credentials;

Create Object Storage credentials

To create a new Object Storage credentials through the Cloud Console UI do the next:

  • go to the Object Storage page and click the CREATE OBJECT STORAGE icon in the upper left corner: st5
  • on the next opened window confirm your intention to create a new Object Storage credentials by clicking the CREATE icon:

st6

The next open page will provide your access secret key and you can save it by clicking the COPY TO CLIPBOARD icon here:

st7

After that, the created Object Storage credentials will be added to the Object Storage page.

Show secret key of the Object Storage credentials

To show again secret key of the created Object Storage credentials you need to do the following:

  • on the Object Storage page identify Object Storage credentials from what you want to see secret key ;
  • click on the Actions icon  ssh9  and select the Show access secret in the list of available options;
  • on the next opened window confirm your intention by clicking the SHOW icon:

st10

The next open page will again provide your access secret key and you can save it by clicking the COPY TO CLIPBOARD icon here:

st11

Re-generate secret key of the Object Storage credentials

To re-generate secret key of the created Object Storage credentials you need to do the following:

  • on the Object Storage page identify Object Storage credentials from what you want to re-generate secret key;
  • click on the Actions icon  ssh9  and select the Re-generate secret in the list of available options;
  • on the next opened window confirm your intention by clicking the RE-GENERATE SCRET icon:

st12

The next open page will provide your new access secret key and you can save it by clicking the COPY TO CLIPBOARD icon here as well:

st13

Delete the Object Storage credentials

To delete the Object Storage credentials  you need to do the next:

  • identify this unnecessary Object Storage credentials on the Object Storage page;
  • click on the Actions icon  ssh9  and select the Delete in the list of available options;
  • on the next opened Confirmation window confirm theObject Storage credentials deletion:

st14

After these steps, the selected Object Storage credentials will be deleted.

Use the Object Storage with the Python API

Python support is provided through a fork of the boto3 library with features to make the most of Cloud Console Object Storage. 

To start use the Object Storage with the Python API do the following:

  • install boto library by running the command:
pip install boto
  • create new file (eg. s3.py ) with the next python code, which will contain three blocks of code for the following tasks:
    • create a connection for interacting with Object Storage API endpoint;
    • created a new bucket with a name that you come up with;
    • get a list of buckets that you own and printed out the bucket name and creation date of each bucket.
vi s3.py
import boto
import boto.s3.connection
access_key = 'your_access_key'
secret_key = 'your_secret_key'

conn = boto.connect_s3(
aws_access_key_id = access_key,
aws_secret_access_key = secret_key,
host = 'upper-austria.ventuscloud.eu',
port = 8080,

calling_format = boto.s3.connection.OrdinaryCallingFormat(),
)

bucket = conn.create_bucket('new-bucket-using-python')

for bucket in conn.get_all_buckets():
print ("{name}\t{created}".format(
name = bucket.name,
created = bucket.creation_date,
))

  • press Esc :wq, and Enter to save the changes;
  • execute your created “s3.py” file:
python s3.py

If the execution was completed without a problem, you should see the next output, what means that we have already created the new bucket, which name is new-bucket-using-python:

st22

So, as you can see, we only have one bucket, which was just created using the Python code above.

NOTE:

As well as Python API you can select other programming languages for using the Object Storage.

Now let's take a closer look at each block of code presented above and deal with additional possible interactions with Object Storage via the Python API:

Creating connection

To create a connection for interacting with the server use next block of code:

import boto
import boto.s3.connection
access_key = 'your_access_key'
secret_key = 'your_secret_key'

conn = boto.connect_s3(
aws_access_key_id = access_key,
aws_secret_access_key = secret_key,
host = 'upper-austria.ventuscloud.eu',
port = 8080,

calling_format = boto.s3.connection.OrdinaryCallingFormat(),
)

Remember to replace the ‘your_access_key’ and ‘your_secret_key’ part with your credential information, and also check if the host is correct - if it matches the correct Region.

Listing owned buckets

To get a list of Buckets that you own use the next block of code:

for bucket in conn.get_all_buckets():
print ("{name}\t{created}".format(
name = bucket.name,
created = bucket.creation_date,
))

It also prints out the bucket name and creation date of each bucket, so the output will look something like this:

st22

Creating a bucket

To create a new Bucket use the next block of code:

bucket = conn.create_bucket('new-bucket-using-python')

The 'new-bucket-using-python' is the name of the new bucket, please enter the desired name for your bucket here.

Creating an object

To create a new object in the bucket use the next block of code:

key = bucket.new_key('hello.txt')
key.set_contents_from_string('Hello World!')

key = bucket.new_key('private_info.txt')
key.set_contents_from_filename('/home/usr/private_data.txt')

So, it creates a file 'hello.txt' with the string "Hello World!" and a file 'private_info.txt' with the data from the local file.

Listing a bucket's content

To list a bucket's content use the next block of code:

for key in bucket.list():
print "{name}\t{size}\t{modified}".format(
name = key.name,
size = key.size,
modified = key.last_modified,
)

It also prints out the object name, size and creation date of each object, so the output will look something like this:

ob10

Changing an object's ACL (access control lists)

To make objects to be publicly readable or to bet private, use the next block of code:

hello_key = bucket.get_key('hello.txt')
hello_key.set_canned_acl('public-read')

private_key = bucket.get_key('private_info.txt')
private_key.set_canned_acl('private')

So, this code makes the object 'hello.txt' to be publicly readable, and the object 'private_info.txt' to be private.

Generating object download URL

For publicly readable objects, we can generate an unsigned download URL. To do this use the next block code:

private_key = bucket.get_key('hello.txt')
hello_url = hello_key.generate_url(0, query_auth=False, force_http=False)
print (hello_url)

The output of this will look something like:

ob7

For private objects, we can generate a signed download URL  that will work for the time period (when the time period is up, the URL will stop working). To do this use the next block code:

private_key = bucket.get_key('private_data.txt')
private_url = private_key.generate_url(3600, query_auth=True, force_http=False)
print (private_url)

Please notice here, when a client application accesses buckets, it always operates with the credentials of a particular user. As every user belongs to a particular project, therefore, for using this links you should add your project ID before your bucket name, and the working links in both examples will look like this:

* for publicly readable objects -"https://upper-austria.ventuscloud.eu/Your_Project_ID:new-bucket-using-python/hello.txt"

* for private objects - "https://upper-austria.ventuscloud.eu/Your_Project_ID:new-bucket-using-python/private_info.txt?Signature=XXX&Expires=1316027075&AWSAccessKeyId=XXX"

Find your Project ID you  can in the link when you use cloud console:

ob8

Deleting an object

To delete an object from the bucket use the next code block:

bucket.delete_key('hello.txt')

It deletes a file "hello.txt" from the bucket.

Deleting a bucket

To delete a whole bucket use the next code block:

conn.delete_bucket("new-bucket-using-python")

It deletes the whole bucket called "new-bucket-using-python".

Use the Object Storage with the client S3 Browser

The S3 Browser is the second way, after the Python API, how you can use the Cloud Console Object Storage and manage data stored in it.

To download and install it on your computer, go to the next page https://s3browser.com/

After installation you can open on your computer S3 Browser and see the next interface of this program:

s-t16

To start work with the Cloud Console Object Storage get access to your Cloud Console account by providing your credential information, for this do the next:

  • on the main Navigation Panel go to Accounts, choose Add new account:

st19

st20

  • on the following page fill in fields as shown below press Add new account.

st21

After these steps, a connection to the Cloud Console Object Storage through the S3 browser is established, and you can see your previously added bucket using the Python API:

st23

Now can add a new bucket or delete a bucket just by clicking on the one of the buttons:

To make bucket public or private go to Permissions tab and select the necessary option:

ob5ob6

So, as you can see, using the Cloud Console Object Storage through the S3 Browser is as easy as using programming languages.