Febooti, Ltd.

CSV · File integrity

Automation Workshop can automate IT and business tasks without writing a line of code. Let's see how we can ensure file integrity when moving them to archive.

Excel CSV file example
Example: Final CSV file

Overview

Files from multiple sources are copied to C:\Input\Files folder. At regular intervals, 4 times a day, Automation Workshop moves folder contents to a unique date-based backup folder. Before moving files to backup their names, sizes, checksums and last modified dates are recorded in a .csv text file to provide means for verifying file integrity later. We will collect data on files before moving them to backup folder.

Solution

Set up the Task Scheduler trigger that will run the Task 4 times a day (once in every 6 hours). The Task will list all files in the C:\Input\Files folder. A subfolder with date and time in its name will be created in C:\Backup folder.

If there are no files, a corresponding record will be written in the no files.txt file in the C:\Backup\Date folder. If there are one or more files, for each of them a series of Action will be performed:

Schedule

First, let's setup a Task Scheduler trigger that will regularly launch the Task with 6 hour intervals. Specify a start date and time and hourly schedule with 6 hours recurrence period.

Task scheduler: recur every 6 hours
Regularly launch the Task with 6 hour interval

Actions

Let's setup Actions for the Task. This is how the Action tab of a fully configured Task looks like:

Task: File integrity CSV
Example: Final task

The information among Actions is passed by Variable Wizard (see underlined elements in the Action tab and green, bold text in parameter fields of Actions).

Notice the indent of Write to File and For each Actions relative to the If and Else Actions. It means that the first Write to File Action will be performed only if number of files in folder is 0, while the second Write to File action will be performed, if the number of files in folder is not 0.

Likewise, the indentation in front of a group of Actions (relative to the For Each Action) indicates that they are subordinate to the For Each action. For Each runs its subordinate Actions for every file it processes (as it processes files retrieved by List Files Action and stored in the List of files variable).

List files

Let's use the List Files Action to retrieve the list of all files currently in the C:\Input\Files folder.

List file Action
List Files Action gets names of all files in the C:\Input\Files folder

Create folder

Let's use the Create folder Action to create a subfolder in the C:\Backup folder with current date and time in its name. We will store both reports and files here.

Create folder Action
Create C:\Backup\Date\ folder

Insert variable after C:\Backup\ text and choose Preset: Sortable file (dashed).

Variable Wizard: Sortable file (dashed)
Preset: Sortable file (dashed)

If

Now we will verify whether there are currently any files in C:\Input\Files folder. If there are none, we will note it in no files.txt file. Else (if there are some) we will process them one by one and store information in the Files-Date.csv file.

Setup the If Action so that it checks if the Number of files returned by the List files Action is equal with 0.

Compare Number of files
Compare Number of files is equal 0

Use Variable Wizard to retrieve the Number of files variable from the List files Action.

Variable Wizard: Number of files
Number of files

No files?

Add the Write to file Action that will write the No files… text in the no files.txt file when there are 0 files listed.

Write to file Action
no files.txt

Let's take the destination path (i.e. Last folder) from the Create folder Action by using Variable Wizard.

Else

Add the Else Action to denote the alternative conditional branch. As specified in the If part, this branch will only be performed when the List files Action returns more than 0 files. See that it has the same indentation level as the If action (i.e., no indent or the leftmost side of the list). Add a subordinate Write to file Action.

To ensure full compatibility with Microsoft Excel, uncheck Save file using Unicode encoding checkbox in options tab.

Write to file Action
Files-Date.csv

Text to write field is organized in comma-separated column that correspond with the order of the following Actions which will provide data for each of them. Note the 3 variables in the configuration of the Write to file Action:

1) Last folder value is taken from the Create folder Action to specify that the report file will be created in the C:\Backup\Date folder.

Variable Wizard: Last folder
Variable Wizard: Last folder

2) Date in Report-Date.csv is taken from the Date variable set to Year / Month / Day format with leading zero added (so that chronological order match the alphabetical one).

Variable Wizard—Date—Year-month-day
Variable Wizard date: year-month-day

3) Number of files variable is taken from the List files Action.

Variable Wizard: Number of files
Number of files

When all 3 variables are combined in Write to file Action, it creates a unique csv file with current date in its name in the previously created folder and writes a comma-separated header row that defines its structure.

Write to file Action
Files-Date.csv

For each…

Now let's fill this csv file with data about each listed file. We will process each file one by one, read its data and store it in the created csv file.

For each
For each loop

When collecting information about listed files, we will run the same set of Actions for each file. The For each Action will perform the cycle by launching subordinate Actions one by one. Add For each Action and use Variable Wizard to load List of files from List files Action.

For each Action
For each file in List of files

For each Action will split a file list in individual files and run a set of subordinate Actions for every file.

Now, let's collect relevant data to fill the csv file. We will use 3 Actions:

Get file information: Current item
Get file information: Current file

Use Variable Wizard to refer to Current item in the For each Action to get information on a file processed in current step of the cycle.

Compute file checksum: Current item
Compute file checksum: Current file

Use Variable Wizard to refer to Current item in the Compute file checksum Action to get hash cryptographic checksum for a file processed in current step of the cycle.

Get date and time: Current item
Get date and time: Current file

Use Variable Wizard to refer to Current item in the Get date and time Action to get information when the file processed in current step of the cycle was last modified.

Once all the three Action have collected the necessary data on current file, let's use Write to file Action to write it in csv file.

Write to file Action
Write file information to file

Using Variable Wizard we will complete a few steps to collect all necessary data together.

  • To indicate csv file specify Last file variable from the previous Write to file Action. Note that append mode should be used rather than overwrite so that each step adds on rather than deletes the previous one.
  • To indicate content to be written in csv file, use Current item from the For each Action, Size from the Get file information Action, Checksum from the Compute file checksum Action, and Modified date from the Get date and time Action. Each variable should be placed in quotes and comma-separated to match the header structure (to guarantee CSV integrity for flawless compatibility with Microsoft Excel).

As a result, the information will be collected for each file in the For each loop (that takes file list from the List files Action) and stored in a unique csv file.

Move to backup

Finally, let's move the processed file itself to the Backup folder (where we stored the csv file).

Let's create a Move file Action and use Variable Wizard for both Source and Destination parameters—for source use Current item from the For each Action; for destination refer to the Last folder in Create folder Action.

Move file Action
In each For each cycle the processed file will be moved to backup

Conclusion

The resulting Task consists of 12 distinct Actions that are performed once every 6 hours. The Actions analyze whether there are files in C:\Input\Files folder and reports every run in a unique date-based backup folder.

Task: File integrity CSV
Example: Final task

When files are found, various data are collected and stored in csv file to ensure backup integrity before moving files to the backup folder.

Ask for more…

If you have any questions, please do not hesitate to contact our support team.

1,286,242
We're growing fast. Thank you!

Video · Monitor an FTP server & send invoice via email

Awesome in automation!
Recent news
  • April 2, 2020
    Automation Workshop v4.1.0 introduces Action favorites and extended view mode. Many minor UX and performance improvements.
  • January 7, 2020
    Automation Workshop v4.0.0 introduces SFTP, FTP, Amazon S3 and WebDAV watchers. 15 new Cloud & FTP Actions. Entirely new and redesigned user experience across entire product.
  • March 14, 2019
    Automation Workshop v3.2.0 ensure smooth For each Action execution. Also, improves Task opening speed.
Automation Workshop has powerful features and almost limitless possibilities under the hood.—Quinn A.
Our clients
Last updated: April 2, 2020
Apr 07, 2020 · 09:53
0.006 · 0.000
0.4 · 3.9