Data Import Service

The purpose of this document is to provide clear guidance to our partners & customers on the rules of engagement regarding submitting data to be ingested into the View Archiving platform. The data submitted would be intended to be ingested into a respective customer’s archive account. These guidelines are enforced by Service Delivery, the team that manages the ingestion/imports processes.

It should be clearly noted, that any data required to be ingested to the service, that does not currently fit in with the ‘standard’ service offering, will be subject to a Statement of Work (SOW), and special pricing included

It is advised that any import of legacy data will come at a charge. Please discuss the applicable charges with your Account Manager.

The standard service offering comprises of the submission of PST, MBOX or zipped EML type data that requires no special extraction or preparation procedures.  Submission of data in any other format not meeting our standard service offerings will be subject to a SOW.

Data submitted, not meeting the standard service requirements, and not previously identified as requiring a SOW will be rejected by Barracuda until an agreed SOW has been met. Barracuda will not work on any data sets not meeting the requirements; thus a perception of delay will be incurred, but not at Barracuda’s expense.

This document supersedes any other document/notice that you may have received in the past.

 
Legacy Data Import Process and Workflow Requirements

Barracuda provides add-on services to allow customers to ingest their legacy email, either from their current mail platform or from previous archiving platforms, into their archive account. The legacy email will be processed through the same secure index and storage process as your daily journal email. To process a legacy data import the following workflow and requirements should be followed to ensure a quality and timely completion of a legacy data import.

*Note - Any deviation of these requirements will result in data being returned to you ordestroyed or will incur additional surcharges.

Overview: Data Import Process

Preparing the Data to be Imported

Barracuda will successfully import a variety of mail file formats and has adopted a standard of preparation that is replicated for the following formats, PST, zipped EML/RFC822, and MBOX. (Any data received outside of these formats will require an SOW prior to importing)

Important: The below requirements are not flexible. They must be followed, or you are at risk of your import data being returned to you or destroyed.

For all File Types (Requirements)

1. No co-mingling of file types in a single directory or folder or compressed file.
2. Supported compression formats: zip, tar.gz, and 7z (anything outside of is subject to an SOW prior to importing)
3. File and path names must contain no spaces or special characters other than @ . _ or –
4. File extensions must be lowercase (.eml not .EML)
5. Eml/rfc822 files MUST be provided in a zipped format.
6. File directory structure within zip files must be no more than 4 levels deep. (File levels are sub-directories within the top-level zip file, and do not represent zips within zips.)
7. Each zip archive provided to Barracuda, should only contain one level of compression. (No zips within zips )
8. The names of zip file(s) provided to Barracuda must include the extension of the file-types contained within. (IE: <yourcompanynam.eml .zip or SMTPaddress. eml. zip or yourcompanyname.pst.zip or SMTPaddress. pst .zip>
9. Compressed file names must contain the company's name or subdomain.

 

Import Do's and Dont's

Do:

• Closely follow the requirements outlined above and throughout this document

 Don’t:

• Send zip files that contain other zip files.
• Send zip files that contain files deeper than four levels from the root.
• Send multipart/split zip files where the sum of all the parts is larger than 250 GB
• Send zip files which contain more than one type of data (for example pst and eml in a single file)
• Include file types within the zip files which cannot be imported into the archive (for example: there should be no loose .pdf or .jpg etc files commingled within your email files.)
• Include spaces or special characters other than @ . _ or - in file names
  (for example not “Sent Items.pst” dsinger(1).zip, or Dave ’ sMail.zip)
• Include spaces or special characters other than @ . _ or - in path names
  (i.e. not \2012 Archive\dec.zip)
• Send uncompressed eml/rfc822 files.
• Place more than 100,000 files into a single zip (i.e. 100,000 individual .eml files in a single zip)
• Do not password-protect pst files.  (Zipped files containing psts, eml, or mbox files may be password-protected.
• Do not send pst files larger than 45 G in size.

*Note - Any deviation of these requirements will result in data being returned to you,
destroyed or will incur additional surcharges.

 

 Special PST Import Guidance

The PST (Personal Storage) file format is used by Microsoft Outlook to store messages on local desktops and network drives.

PST files may be generated either directly by your users OR via an automated application such as exmerge, which extracts messages from your Exchange Server into a collection of pst files.

**A conversion process takes place during the import which converts all messages within the PST file to a raw data format. It is recommended that, while searching for a user's legacy data within the archive, both the user's name/alias and their SMTP address are used.

We recommend that individual uncompressed PST files sent for import should be no larger than 45 GB in size. Barracuda will attempt to import larger PST files but, in general, PST files larger than 45 GB are more likely to be corrupt or to encounter errors during the conversion to eml format which precedes import. This can cause a delay in the import process or an incomplete set of messages in the archive.

When sending PST files for import, the PST files themselves must not be secured with any password protection

The PST files may be encrypted in a password-protected 7-zip file (see Compression & Encryption below) and the 7-zip files may be copied to a Snow device for shipment or uploaded to a site for Barracuda to download and import into the archive.

When using 7-zip for PST files, please create a separate 7-zip file for each PST file.

Compression & Encryption

We recommend the use of 7-zip for compression and strong AES-256 bit encryption. 7-zip can be downloaded at no charge. ( http://www.7-zip.org/ ).

The illustration above shows a folder that you have locally/on your network containing a volume of files. These files can be .eml/rfc.822 or pst, but each zip file should contain files in a single format type -- not files with different formats within a single zip file. The name of the zip file should identify the type of files contained - for example companyname eml .zip or username@mycompany.com. pst .7z.

 To encrypt using 7-zip

• Open 7-zip File Manager
• Browse to location of files to be sent for import
• Once you have successfully browsed to the folder, highlight all files to be included in a single zip file
• Next click File from the menu items, then click 7-zip , and then click add to archive
• Change the compression level to fast
• Enter password if encryption is desired ( When encrypting files for an import, all files for a given import ticket must have the same password.)

 How to get the data to Barracuda

There are two methods to deliver the data to Barracuda.

1. Upload the data to a site and provide Barracuda with credentials to download the data. Some examples of sites which can be used are your own FTP site, Box.net, G-Suite, O365 One Drive (included with O365), Amazon s3, Azure or Google Cloud Storage. This is the fastest and least expensive way of sending the data and is particularly suitable for small and medium sized imports.
2. Open a support ticket with Barracuda to arrange for AWS to ship a Snow device, which is the property of AWS, to the location where the data is present on the network. You will have a specified number of calendar days to copy the data to the Snow device and turn the Snow device over to UPS for return shipment to AWS.
   There is a surcharge for shipping the data on a Snow device in addition to the usual per gigabyte import charge. There is also an additional daily charge if the Snow device is not turned over to UPS for return shipment within the specified number of calendar days of receipt. It is necessary to connect the Snow device to your network, assign it an IP address, and transfer the data with a particular software application.

 How long will my import of data take?

Processing data for ingestion can take a varied amount of time to complete. This KB article is provided to help you to understand your possible options for improving the time frame to import.

Importing Legacy Data items is a common everyday process here at Barracuda and our cloud optimized infrastructure is designed to provide the best value of service for everyone!

When Barracuda conducts a process to import data for any client, there are many factors that will need to be considered that can affect how long an import process will take to complete. Some of these factors are, Number of individual PST, or EML folders overall, size of a PST files, complexity of a folder structure housing the messages, number of documents and number of messages.

All of these factors are taken into consideration when we process your data. Other factors that are taken into consideration are: extracting compressed / encrypted data, data conversions, and securely storing, encrypting and indexing all the files.

When we are ready to process your data for import, we assign a set number of resources to churn through the data. The number of compute power resources utilized are aligned to the standard import costs that have been provided to you.

Guideline Time frames for Import

Completion of Import

Barracuda will update the ticket confirming the import has been completed and the uncompressed size of the total import. The support ticket will then be marked as resolved but will not close until 3 days after completion date.

It will be expected that, should  there be any discrepancies with the data imported, that these be reported to Barracuda Service Delivery within 30 days of the import completing. After this period, it will be harder to help identify/rectify any issues.

 

 FAQ’s

Why do some imports go faster than others?

• Once files are converted into a standard MIME/RFC.822 format they are inserted into a queuing service for secure archiving and indexing. The processing rate varies at this stage is based upon a few factors.
  • Size/profile of email attachments that go through a text extraction service.
  • Availability of excess computer services.
  • Capacity consumed by day-to-day archiving activities (daily journaling, exports, other import tasks, etc.)
  •  

What can be done to help improve the efficiency of importing this data?

• Supply messages in compressed EML format. The most efficient method to process any email data will be to provide this to Barracuda as a standard MIME/RFC.822 format (EML).
• EML files must be compressed before being submitted. The most efficient size for the zip files is 25 G or less.
• When data is submitted as PST files, we recommend that single PST files do not exceed 45GB and are proved not to be corrupt if attempted to open via an outlook client.
• PST Files should NOT be password protected. Password protected PST files cannot be processed by the service and may require that the data is re-submitted without any passwords. (It is fine to encrypt 7-zip files that contain PST files, but the PST files themselves should not be encrypted.)
•  

Can Barracuda speed things up?

• If there is a need for you to provide data that has urgent e-discovery needs then YES, the timeline for importing your data can improved by utilizing additional/dedicated compute resources at an extra premium cost to you.

 

Backup considerations

• Ensure you that you retain a copy of any data you supply to Barracuda, whether exported from your mail servers or standing legacy data from a file store.

 

Costs of Imports

• Barracuda charges a very modest, one-off fee for the import of data. The fee is calculated on size of the data supplied, that being a per GB charge for the data in an uncompressed state. If you have data that needs to be imported, you will be able to estimate the cost of the import when communicating with your service providers. There will be additional surcharges when the data is shipped on a Snow device or when the format of the data does not meet the standard requirements described in this document.

What happens to my mail files which I have supplied to Barracuda?

• The data supplied will be stored securely on Barracuda’s archive infrastructure, in readiness for processing.
• The data will be staged and ingested into your archive account. It will then be indexed into the archive as redundant copies of the data are made automatically. This indexing process makes the data searchable in the archive and the redundant copies ensure high availability of the data.