Monitor a folder in AWS S3 bucket for changes with Amazon S3 Watcher

Automation Workshop features a remote file monitoring module—Amazon S3 Watcher. It scans a folder in AWS S3 bucket for file changes. It does not require AWS Transfer for SFTP Service to be enabled in the AWS Console, however it is possible to monitor such Amazon Simple Storage Service that has the SFTP Service enabled using SFTP Watcher.

Amazon S3 Watcher can launch a Task when a file or folder is created, modified, deleted in the remote bucket. It can also watch for the number of files in a directory, and look when a file reaches a certain size limit. A file mask filter allows monitoring for selected file types or files with specific names.

Amazon S3 Watcher Trigger is part of a Windows Automation solution—Automation Workshop. It allows monitoring AWS S3 on your premises (self-hosted solution) without depending on online services.

Automation Workshop has built-in more than a hundred Actions that can do a multitude of tasks on your files, such as download or upload files, synchronize whole directory trees, compress files, send emails, and much much more.

Features & highlights

Watch files and folders for modifications on a remote Amazon S3 server.
Monitor when new files are created in AWS S3 buckets.
Trigger on file events like file modified or file deleted.
Auto-reconnect mechanism ensures stable data transfers even in case of dropped connections.
Monitor for file size to reach the defined amount.
Watch for any file event on Amazon S3 storage, or on Amazon cold storage service Glacier.
Easy to use user interface with a very short learning curve.
Allows actively monitoring an AWS S3 folder in the S3 bucket, and downloading or uploading files, all without knowledge of AWS CLI, Lambda, or other scripting.
Scan a folder or bucket for the total number of files and launch a Task when the count is reached.

Usage examples

Amazon S3 Watcher allows monitoring Amazon Simple Storage Service for new files in the directory, and to download them automatically, when such a file appears.
Monitor AWS S3 bucket in real-time, and as soon as changes occur, automatically sync files from one AWS S3 bucket to another S3 bucket.
Allows automatically downloading new or changed files from AWS S3 to the local PC or server with an automatic Task, that is easily created in GUI editor, without writing a single line of program code.
It is possible to scan an Amazon S3 bucket for files that exceed a certain size, to control the cost of the storage service, by automatically sending email alerts or reports, when specified size is surpassed.

Documentation

Directory · Watch a remote Amazon S3 storage for changes in files and directories.
Conditions · Select what kind of changes to watch for, such as new, existing, deleted, or changed files. Also, watch for large files or the number of files in a directory.
Options · Set how often Amazon S3 Watcher will check files in the remote Amazon Simple Storage Service server.

Dynamic variables & Events of the Amazon S3 Watcher.
Variable Wizard and dynamic values of all the File & Folder Actions for local and network drives, and remote FTP, SFTP, WebDAV, Amazon S3, Google Cloud Storage, and DigitalOcean Spaces servers.
Streamline automation processes by connecting Triggers and Actions using Variable Wizard.

S3 compatibility

The Amazon S3 Watcher Trigger allows automated workflows that watch other S3-compatible cloud platforms. Monitor S3-compatible APIs for new files, for file modifications, as well as removed files and directories. The Synchronize Directories Action allows to sync directories between multiple cloud providers using a local server as a proxy.

Backblaze B2 Cloud Storage is fully S3-compatible.
OpenStack Object Storage (swift) is fully S3-compatible.
Dell EMC ECS Enterprise Object Storage offers rich S3-compatibility.
The Oracle Cloud Infrastructure Object Storage service is S3-compatible with some minor tweaks required.

S3 more

There are at least two S3-compatible storage platforms that can be self-hosted on-premises.

Cloudian—Exabyte-scalable and cloud-compatible, Cloudian software-defined storage.
Eucalyptus—Open source software for building S3-compatible private and hybrid clouds.

Tutorials

Discover

Automation Workshop includes many more awesome Triggers and numerous Actions to aid you to automate any repetitive computer or business task by providing state-of-the-art GUI tools.