FMS Transfer for Content API Clients

Direct Multipart File Upload

For smaller payload files, we recommend using direct multipart file upload method. Please note the following limitations of this method:

• The maximum file size supported for direct upload is 200M, but we recommend using direct upload only for files under 1MB in size. For larger payload files, please use TUS protocol as described later in this page.
• Upload can not be resumed or paused. In case of interrupted transfer, the client should invoke POST​/jobs​/{jobId}​/sourcefiles again to obtain a new transfer object.

Upload files to Content API is a two-step process:

Step One

Call 'POST /jobs/{jobId}/sourcefiles', specifying the fileName in parameter. Content API returns FMSTransferObject which includes a fmsPostMultipartUrl field.

Step Two

Make an HTTP POST request to the fmsPostMultipartUrl to upload your payload file.

Direct Upload Code Sample

C#

		var uri = new Uri($"{fmsPostMultipartUrl}");
		using var form = new MultipartFormDataContent();
		using var fs = File.OpenRead(fileName);
		using var streamContent = new StreamContent(fs);
		using var fileContent = new ByteArrayContent(await streamContent.ReadAsByteArrayAsync());
		fileContent.Headers.ContentType = MediaTypeHeaderValue.Parse("multipart/form-data");
		form.Add(fileContent, "file", Path.GetFileName(fileName));
		var response = await httpClient.PostAsync(uri, form);
 		

Angular

		const url = `{fmsPostMultipartUrl}`;
		const formData = new FormData();
		formData.append('file', this.file);
		await this.httpClient.post(url, formData).toPromise();

File Upload With TUS Protocol

For larger payload files, we recommend using TUS protocol to upload them. TUS is an open protocol that supports resumable uploads. Please see tus.io for more details.

Upload files with TUS protocol is also a two-step process:

Step One

Call 'POST /jobs/{jobId}/sourcefiles', specifying the fileName in parameter. Content API returns FMSTransferObject which includes a fmsUploadUrl and a fmsSASToken.

Step Two

Using a TUS client to upload the file to fmsUploadUrl. Please note that TUS is a protocol, you can implement your own TUS client if an open source TUS client is not available for the language of your choice, or if available open source TUS client is not suitable for your need.

Note: The upload endpoint expects a ‘Content-Length’ header. Some TUS clients may not include this (notably Java client). Please see Java code sample below.

Upload Code Sample

C#

			var info = new FileInfo("somefile.json");
			var tusClient = new TusClient();
			tusClient.AdditionalHeaders.Add("sastoken", fmsSASToken);
			var tusMetadata = new (string key, string value)[]
			{
				("name", info.Name),
				("contentType",MediaTypeNames.Application.Json),
				("chunkSize", $"{1024 * 1024 * 1}")
			};
			var fileUrl = await tusClient.CreateAsync(fmsUploadUrl, info, tusMetadata);
			var uploadOperation = await tusClient.UploadAsync(fileUrl, info, chunkSize: 1D);
		

Java

Note: Java TUS code snippet only works when Content-Length header field is set in the TUS HTTP requests. To ensure that, please set the system property "sun.net.http.allowRestrictedHeaders" to "true". You can do so either by setting it as part of JVM command line:

-Dsun.net.http.allowRestrictedHeaders=true

or by setting it in Java code:

System.setProperty("sun.net.http.allowRestrictedHeaders", "true"))

 

			Map headers = new HashMap();
			headers.put("sastoken", fmsTransferObject.getFmsSASToken());
			headers.put("Content-Length", "0");

			final TusUpload upload = new TusUpload(file);
			TusClient client = new TusClient();
			client.setUploadCreationURL(new URL(fmsRedirectUrl));
			client.setHeaders(headers);
			client.enableResuming(new TusURLMemoryStore());

			Integer chunkSize = 1024 * 1024 * 1;

			Map metadata = new HashMap();
			metadata.put("name", file.getName());
			metadata.put("chunkSize", String.format("%d", chunkSize));
			metadata.put("contentType", contentType);
			upload.setMetadata(metadata);

			log.info("Starting upload...");
			TusExecutor executor = new TusExecutor() {
				@Override
				protected void makeAttempt() throws IOException, ProtocolException {
				TusUploader uploader = client.resumeOrCreateUpload(upload);
					uploader.setChunkSize(chunkSize);
					uploader.setRequestPayloadSize((int)file.length());
		
					do {
					long totalBytes = upload.getSize();
					long bytesUploaded = uploader.getOffset();
					if (totalBytes == bytesUploaded) {
					break;
					}
					} while (uploader.uploadChunk() > -1);
		
					uploader.finish();
					log.info("Upload finished.");
					}
				};
			executor.makeAttempts();			
		

File Download

The download methods on the API require clients to follow redirect (HTTP code: 307). Apart from that, no special handling is required.