Remote administration of connected nodes is passed through Central's interface or in the case of applets passively over Central's TCP port. Usually, no special configuration is needed on the server side to form this passive connection. FileCatalyst Clients typically requires no network configuration to form passive connections to the Server for transfers , nor to the console for reporting status, alarms, and other information.
Installing FileCatalyst Central Note: FileCatalyst Central may be deployed on the same machine as any other FileCatalyst application, but work most effectively when installed on a separate machine to independently monitor the health of all other FileCatalyst nodes. You will also need to trust the installer application. Changes to the configuration file if required — see guide below come into effect only by stopping and restarting the service. Instructions on how to accomplish this can be found in the.
If you are upgrading from FileCatalyst Monitoring Agent, you can preserve your records and settings by copying the. Upgrading FileCatalyst Central Windows Download the latest installer contact your account representative for details Stop the FileCatalyst Central service either from Services manager or from the shortcut found in Start menu or install path Run the installer to install "in place" of the existing FileCatalyst Central. Linux If you are running FileCatalyst Central as standalone e.
Restart the FileCatalyst Central application either as a service or standalone If you are running as a service and have upgraded from a version older than 3. Linux Note: You may need admin access to perform some of these commands If you are running the FileCatalyst Central stand-alone, shut the instance down: Open a new Terminal window Navigate to the install path e. Preparing FileCatalyst Deployment for Monitoring After installation, you must perform an initial web-based configuration to put the console UI online and make dispatchers available.
Configure FileCatalyst Server Monitoring is disabled by default. With "Enable remote monitoring" set to true, a number of other fields configure the connection to Central: Enable remote monitoring Enable and disable the connection between FileCatalyst TransferAgent and one instance of FileCatalyst Central.
Port The port at which FileCatalyst Central is found. Alias A name of your or your administrator's choosing, which will allow FileCatalyst Central administrators to easily identify your TransferAgent.
Notify on transfer success Send a notification to FileCatalyst Central upon each transfer success. Notify on transfer success Send a notification to FileCatalyst Central upon each transfer failure. Notify on transfer success Send a notification to FileCatalyst Central upon each cancelled transfer. FileCatalyst Central Interface: Conventions This section of the guide will walk through each available view of the application, describing the information and functionality exposed in each view.
Overview The admin sub-menu allows you to access the following: Documentation this document Configuration FileCatalyst Central configuration.
Note: this pauses the continuous nature of the updates, but filtering and sorting will retrieve new up-to-the-moment data for the table or chart in question. Columns available for sorting show arrow widgets Search box allows the user to quickly isolate only the data they wish to view. Note: some columns of certain tables are not currently searchable with the text filter; for example, "Severity". The tables and graphs are wrapped in sortable panels. Depending on your resolution, the panels can be sorted into areas that fit their size.
That means panels on the left side cannot be moved to the right side. Also, the new sort is not permanent; after reloading the page, the panels will return to their original positions.
Currently, only Internet Explorer does not support this feature. Tables are also resizable vertically. The tables will return to the original size once the page is refreshed. About FileCatalyst and FileCatalyst Central It is a pop-up menu that will display itself in the current rendered page, see image below. The About FileCatalyst pop-up contains information about installation environment and version, as well as three buttons: View Logs This button opens a new page displaying the FileCatalyst Central logs.
Diagnostic Report The Diagnostic Report button collects state, logs, and configuration information about your application that will be useful to FileCatalyst engineers for analysis and troubleshooting.
Classic View Panels Transfers : transfers currently active on the system Current Bandwidth : a graphical representation of the current bandwidth of the selected server node. Nodes may be selected from the drop down. Alarms : the ten most recent alarms.
To view more, go to the dedicated Alarms page Nodes : a list of nodes which Central is aware of. Offline nodes will appear here unless they are "detached" in the Nodes view. Transfer Statistics : aggregated statistics for the volume of data transferred by last hour, day, and week. Filename: The filename minus full path being transferred Status: Specific activity occurring for the transfer task Progress: Progress bar representing estimated percentage of task completion Bandwidth Usage Select which node's bandwidth usage will be shown.
Disconnected nodes are grayed out. Gray line: total bandwidth used on the selected node Orange line: total outbound bandwidth from the node Blue Line: total inbound bandwidth to the node Alarms Overview The columns are a subset of the full details seen in the Alarm Details view.
FileCatalyst Central Interface: Users and Groups FileCatalyst Central allows the administrator to create, modify, delete, disable, or enable users of the system. Add : Pressing the "Add User" button in the header of the Users table invokes a dialog. Fill the required information and press "Save" to add a new user to FileCatalyst Central. Edit : Click the gear icon and select "Edit" to open a dialog.
This dialog is similar to the "add" dialog, allowing the administrator to change the user's details or password. Delete : Click the gear icon and select "Delete". Press "OK" to delete the user. Assign a role : Select a user from the Users table to see its assigned role. The assigned role is marked by a checkbox. The checkboxes are also used to update the user's role, and take effect without further confirmation.
Possible roles: Full Admin : User has full administrative privileges; user may interact with Central and all connected nodes in order to change settings and to trigger actions such as initiating a file transfer.
Admin All Nodes : User may trigger actions on assigned nodes, but may not change settings. View All Nodes : User may view Central and assigned nodes, but may not trigger actions or change settings. Assign a group : When a user is selected, they may be linked to or unlinked from a group. A group extends its permission for a given node to all users in that group. In other words, a group with higher permission levels may elevate the user's permissions for as long as they belong to the group.
When multiple groups modify permissions, the highest permission level will apply. Nodes visible by the selected user via its groups will appear on the "Nodes" table. System administrators should take particular note of this table's "Permission" column, using it to confirm that users have the correct access level for individual nodes.
Group assignments On the Group assignments tab, the administrator can create, edit, delete, or disable a group, as well as link particular users and nodes to a group. Add : Press the "Add Group" button on top of the "Groups" table to invoke a dialog.
Fill the required information and press "Save" to add a new group to FileCatalyst Central. Edit : click the gear icon and select "Edit" to open a dialog allowing the administrator to change the details of the selected group. Delete : click the gear icon and select "Delete". Press "OK" to delete the group. When a group is disabled, related user permissions are no longer applied. Assign a user : The administrator may link or unlink a group by using the checkboxes in the "Users" table.
Users linked to the group will inherit properties from the group, such as nodes assigned to the group. Assign a node : The administrator may link or unlink the group to a node. The "Nodes" table has two checkbox columns: "Assigned" and "Read and Write". By marking the "Assigned" checkbox and leaving "Read and Write" unmarked, the group will have read-only permission to Server and HotFolder nodes.
By marking the "Read and Write" checkbox, "Assigned" will also be marked automatically, meaning that group can also manipulate the node. If no checkbox is marked on the Nodes table, the group will not have any permisson over the node. Node assignments On this tab, the administrator can link groups as well as inspect which users have access to individual nodes.
Assign a group : After selecting a node, the administrator may modify its relationship to the groups. The "Groups" table has two checkbox columns: "Assigned" and "Read and Write". By marking the "Read and Write" checkbox, "Assigned" will also be marked, meaning the group can also manipulate the node.
If no checkbox is marked on the "Groups" table, it means the node and group are not related. User permissions : once a node is selected, the Users table will render any user that has any privileges on the selected node. Note: When multiple groups modify permissions, the highest permission level will apply.
The syntax is similar to the nslookup switch on Unix systems. By clicking on the icon on the right side of its text box a dialog will appear which allows the administrator to sort the query authentication priority.
Adding FileCatalyst Central authentication : if the "CENTRAL" service representing authentication via Central's own internal database has ever been removed from the list of directory switches, it can be added back by clicking the Add Central Authentication button which appears next to the New Directory Service button. Enable Auto-provisioning : If an OpenLDAP or an Active Directory has been set as an authentication service, the administrator can configure FileCatalyst Central such that first login automatically creates a Central database user based on the entry in the remote authentication service, but using settings listed under the User Defaults tab.
If this is disabled, users must be explicitly created by an administrator before being granted access to Central. Test Selected Service : This tests the selected directory service ie. The logs created during the authentication process are displayed on-screen to help trace any errors that may occur. User defaults This panel contains the Groups , Nodes , and Roles tables. Current Transfers A continuously-updated view of the ongoing individual file transfers on the managed FileCatalyst system.
Administrators may click the alias to open a separate window with a remote administration view for full control of the node. Username: the name of the user account on the FileCatalyst Server.
Filename: the full filename plus extension s being transferred. Start Time: Day and time the file transfer was initiated. Status: The current status of the file transfer. Transferred: amount of data transferred.
Rate: current effective transfer rate in Kbps, Mbps, or Gbps according to which is most appropriate. Session History This view shows an ongoing history of transactions with definable start and end times.
Administrators may click this ID to see the individual file transfers that comprise the session. Search Records FileCatalyst maintains a database of transaction history. To view past transactions, you may search the database based on certain criteria using the form: Result Set: Whether records should be retrieved at the session level all files transfered within a session matching the search or the file level an individual row for each file that matches the search.
Status: Narrow results down to successful, failed, or canceled transfers. Report The report page is responsible for displaying aggregated transfer data collected by the FileCatalyst Central instance. The second container holds the nodes that are being filtered on.
Entering any text into the field will filter out those nodes that don't match the term entered. The nodes that are filtered out still exist; they are simply visually hidden.
Emptying that search field will reset the filter. The Report panel contains the histogram representing the amount of data transfered to and from the filtered nodes. If no nodes are selected as filters, it shows the information relative to all available nodes. Below the histogram is a summary table showing the same information as the histogram, but as an alpha-numeric representation.
Nodes Nodes are any instances of FileCatalyst or 3rd-party applications communicating with one-another within a FileCatalyst deployment. This column shows the highest severity of any one alarm raised by the node Sessions: the total number of separate transfer tasks ongoing to and from this node Transmit Rate: total rate for upload from this node, in the most appropriate unit of measurement Kbps, Mbps, Gbps Receive Rate: total rate for download to this node, in the most appropriate unit of measurement Kbps, Mbps, Gbps Tools: column containing tools to assist managing the nodes.
Current tools include: Detach broken chain icon : Remove this node from being monitored by FileCatalyst Central. Detached nodes may reappear after they are restarted or after a configuration change on the node side. The detach function is typically used to remove decomissioned nodes from the Central view; an administrator will not normally detach a node which is expected to be active again. Refresh refresh icon : The files related to the node's view may be unavailable or corrupted.
Clicking "refresh" will delete the node's HTML files and then attempt to recover a new copy from the node. This option is only available when the node is active, and the process may take a few seconds or several minutes depending on network conditions. Current Alarms All alarms have multiple values assigned to them to better understand the fault and the service raising the alarm.
Some of these details are already visible in the table row itself, but the expanded view provides the full text for each, along with some details not otherwise available: Service Type: the type of FileCatalyst product raising the alarm eg. Items in the severity column are also color-coded based on severity, with bold red being the most severe.
Note : discarding an alarm does not necessarily permanently remove it. If a node continues to have a communications error, for example, the alarm will be re-raised at the set interval. Notifications will not re-appear in this view once discarded. Users: Lists all users on the server, and then presents information about each, including their name, home directory, and file transfer limitations.
Virtual Folders: Lists all virtual folders on the server and information about each, including the local filesystem path, which users have access to the folder, and which groups have access to the folder. Groups: Lists all groups on the server and information about each, including which users belong to that group, and which folders the group has access to. HotFolder Nodes When you click on a FileCatalyst HotFolder node name on almost any table view, you will be brought to an information page for that particular node: Tasks: Details for any ongoing current transfers, at the task rather than session or file level.
Tasks may be executed or canceled. HotFolders: view the names and local paths of HotFolders configured on the node. Sites: view the names and hostnames of sites configured on the node. If a map already exists, you can switch to it using the select box. The star icon is used to set the selected map as favorite. The favorite map will be the default map in the Map and Data view.
The Map and Data current map button allows switch between editing maps and the Map and Data view. You must save your changes before switching to the Map and Data view, or they will be lost. The Save Map button saves the current state of the map.
The Add Map button begins the creation of a new map. The Delete Map button deletes the currently selected map. The Add nodes button invokes a dialog allowing the administrator to select nodes and add them to the map. The Background Image button invokes a form responsible for adding or selecting a background image, as well as modifying related properties. Edit Map: Nodes When Add Nodes is clicked, the dialog shown on the image above appears, containing the nodes connected to your instance of FileCatalyst Central.
Once you selection is done, click Add Selected Nodes The Cancel button will close the dialog and cancel node selection. Clicking the icon at the top-left corner of each node will give the option of see more details about the node or remove the node from the map.
Note: removing a node from a map will not detach the node from FileCatalyst Central. Edit Map: Organizers When Add Organizers is clicked, the dialog shown on the above image appears, containing a form for adding a new organizer. The following fields are available when creating a new organizer: A title or description which will be visible on the organizer on the map. An initial width you may resize later for the organizer, with a minimum of pixels.
An initial height you may resize later for the organizer, with a minimum of pixels. After adding organizers, the map may look something like the following: The organizer on left-top corner shows all default options no title, px by px.
Three pre-installed options are available; you may click Apply to select one for the current map. Upload a new image pressing the browse button, which will call the file explorer implemented by your browser. Once the image is uploaded, it will become part of the gallery of images and will be available to Apply. After Apply ing an image, click Done button.
You may also Cancel the dialog at any time. Background Repeat allows smaller images to repeat: repeat-x for horizontal-only, repeat-y for vertical-only, or repeat to tile in both directions. When No-repeat is selected or the image is bigger than the canvas, there will be no tiling. Opacity allows the background image be partially transparent. Height allows the background-image stretch vertically. Width allows the background-image stretch horizontally.
Remove Current Background Image removes the background image from map. This also has the effect of removing these controls. Resize Background To Fit will stretch the image until it fills the container width. Height is set proportionally.
Reset Background Position moves the background image to the default position aligned top-left. The map must be saved for this change to persist. Cancel to close the dialog without any changes. Success: Failure due to overlapping nodes: Failure due to overlapping organizers: Other error messages may also appear for unexpected situations. On this page, a number of UI user interface conventions are followed: All panels may be collapsed with the icon to the left of the title.
Also, if not using Internet Explorer, they can be rearranged vertically to suit the administrator's needs. A select box above the canvas provides the possibility to switch the map in the event that two or more maps have been created.
The default map if defined is marked with a star filled with blue. The double-screen icon creates a second view of the filtered data see Map and Data: Split Screen for more details The Edit Map button appears for admin users, allowing you to switch to the Edit Map view.
The Report from Selection button sends the user to the report page, with the report pre-filtered based on the map selection. The Alarms table shows a summary of the current alarms based on selection.
The Transfer Histogram presents data transfer information about the canvas selection. It contains a select box with predefine timeframes to show the related data. The Tasks shows the aggregated tasks from all HotFolders in the map selection. The Sessions table shows aggregated session information from the current map selection.
If an individual task is selected, sessions are filtered by that task. Map and Data: Selectable Nodes In the Map and Data view, the nodes cannot be dragged and must be re-positioned in the edit view; however, they do present certain options to the administrator.
When selected, a node's border turns blue. All selected nodes act as filters for the page's panels. When the menu icon is clicked, a popup menu appears. This option may not be available if the node is not connected to Central. An icon will also appear in the upper-right corner of the node widget when there is an alarm raised by the node.
See the next section for details. Map and Data: Nodes Alarms As mentioned in the previous section, if a node has an alarm, an alarm icon is displayed on top-right corner of the node. Map and Data: Organizers In the Map and Data view, organizers cannot be dragged and must be re-positioned in the edit view; however, they do present certain options to the administrator.
A selected organizer has a blue border to identify its state. The contents of selected organizers act as filters for the page's panels. The filter icon must be clicked to toggle selection. Clicking the border will NOT select the organizer. Map and Data: Links Links are dynamically generated. Map and Data: Making Selections The tables, bandwidth usage graph, and transfer histogram have information filtered primarily by the selections on the canvas. Selections may be made and modified with the following user interactions: Single click selection Clicking on a single node, organizer or link will remove the selection of all other nodes, organizers or links, leaving only the last clicked object selected.
Clicking on an empty area on the canvas will de-select all objects. Links and organizers are not selected. Selecting using control key If the Control Key is held while the user is clicking a node, organizer or a link it will added the item for the current selection. If the item is already selected, the Control-click will de-select it. Map and Data: Multi Selection When the selection involves different type of elements, e. Near the title of each table, a list of currently-filtered nodes will appear: Organizers without name will be represented as "organizer - id" , where id is the numerical identifier of the organizer Some of these tables' titles link to related information.
Transfer Histogram The histogram will show aggregated information from the current selection. More Tag If three or more items are selected, two of them will be presented, and the difference number of the selected minus the 2 already showed is represented with a " more The title of the Last Files table links to the Search page. Map and Data: Transfer Histogram The select box shows the time ranges for filtering the data. The Today option shows data from a. The title Transfer Histogram: is presented together a list of the elements selected.
The blue bars represents the amount of data Received, while the orange bar represents Transmitted. Below the select box, a subtitle shows what the chart currently represents in terms of data unit eg.
Leaving the mouse over a bar shows a box detailing the time and amount of transferred data. By Session The rows of the Sessions table can be used as a filter by clicking on them. Removing Row Filters It is possible remove the filter by making a new selection, clicking on the selected row again to de-select it , or clicking on the filter icon beside the filtered table label. Map and Data: Split Screen The double-screen icon when clicked launch a new tab or window depends on your browser settings.
Filters The table filters work for both direction in the split screen, which means that you can filter or remove filter clicking in the table on the main screen or splited tab. Filter Applied on left tab affecting right tab: Remove Row Filters One tab to another It is the same of the previous section Map and Data: Filtering Table Results , it is possible remove the filter doing a new selection, clicking on the selected row again or clicking on the beside the filtered table label.
FileCatalyst Central Interface: Configuration The configuration page contains a number of panels for configuring various aspects of a Central deployment. Server Configuration: License This tab has two purposes: first, it provides all information about the current license; next, it allows the administrator to set a new license key. Version read-only : This full string should be provided to FileCatalyst should customer support be required.
Request String read-only : when applying a license renewal or upgrade, the FileCatalyst representative will need this string. License Key read-only : the current license key. Typically for information purposes only License Details read-only : a list of all privileges and limitations tied to the FileCatalyst Central license. Change License Key: click or tap this button to launch a dialog box with a text field for the license key provided by FileCatalyst.
Enable Email: check this box to allow Central to communicate with an SMTP server From: email address to be used as the "sender" To: recipient of automated emails. Multiple addresses may not be specified in this field; to send to multiple people the administrator must create an address which has been set up to forward to multiple recipients. If only one other recipient is required, use the "cc" field. If this is the case, check this box and complete the "Username" and "Password" fields. Username: the username required for authorization on the mail server Password: the password required for authorization on the mail server.
Server Configuration: Alarms Settings A number of options allow the administrator to tailor the behavior of capturing and reporting alarms: Polling Interval: The interval at which Central will check with connected nodes for alarm status. In theory, an alarm could be raised and solved within the polling interval and therefore never be exposed to the interface. Notification Interval: How often the current list of alarms is updated for rebroadcast purposes.
Alarm Retention: Amount of time resolved alarms are kept in a viewable history. Session History Retention: Amount of time completed sessions are kept in a viewable history. Active Alarms: a list of possible alarms that may be triggered during the polling phase all available alarms enabled by default Disabled Alarms: a list of alarms that will be ignored for notification purposes.
Administrators will not be alerted by email for disabled alarms. For example, an admin may not want to be sent an email for every Transfer success, failure, or cancellation. Please note a disabled alarm will still display under Alarms section of Central.
Username: remote applications will use this name for authentication Password: remote applications will use this password for authentication New Password: to update the remote authentication password, enter the current password into the "Password" field, a new one into this field, and a confirmation of the new one into the "New Password Re-Type " field. New Password Re-Type : to update the remote authentication password, enter the current password into the "Password" field, a new one into the "New Password" field, and and a confirmation of the new one into this field.
Select the IP which can be reached from remote connections. Connection Port: set the port for establishing remote connections for monitoring. Use External IP: Enable this option if the applications connecting to Central are to use an external IP which will be forwarded to the Central machine. Local Configuration Values This panel is used to configure how the local browser interacts with the application.
Idle Timeout minutes : the amount of time the web application interface must be idle no mouse or keyboard entry before updates are paused. Once paused, there are no further HTTP requests and the web session will eventually time out. Web session timeouts are set by the web server and are not configured via this interface.
Session Timeout seconds : time allowed for fail communication between the browser and FileCatalyst Central before used logged out. Also, it will clear sorting, search and pagination cache for tables within the application. Admin Tools This panel is initially collapsed, as it contains tools that should not ordinarily be needed.
Rebuild History: Rebuilds and replaces in-memory statistics with the persistent statistics stored in the FileCatalyst Central database. The application itself is not at risk by rebuilding history; however, the operation will heavily consume system resources as the rebuild happens; depending on system specifications, this operation should complete in under a minute.
The amount of information available will depend on the Session History Retention setting currently under the Alarms Settings configuration tab. The Configuration File All of the configuration options available through the user interface are accessible through the FileCatalyst Central's configuration file, maconfig.
Making Modifications The only reliable way to work with this file is: Stop the FileCatalyst Central application from running. In most cases, this will mean stopping the service. Modify the configuration file using the editor of your choice Save the configuration file Run the FileCatalyst Central application.
This will usually mean restarting the service. If full path if not indicated, a relative path is used where JVM is launched. Parameters Only Found in maconfig. Windows service configurations: a collection related to how the service behaves in Windows. Among these are heap size parameters wrapper. Please uninstall the service before modifying, and then reinstall the service once complete.
System Properties Used for debugging purposes to set advanced system properties. In networks with high amounts of latency, enabling this property will reduce the amount of time that it takes for web browsers to load the web-based resources after they have been successfully loaded for the first time.
If the property is disabled, then all remote HTML interfaces will be stored based on the version of the node. If you are experiencing issues with the amount of drive space that Central is using on your system, disabling this property may reduce the amount of space that is needed for storing remote HTML user interfaces. This feature blocks SSL connections to the FileCatalyst server from being established if the server certificate is not trusted by your client.
This feature is only applicable when SSL is enabled. When this is set to true on your client, the SSL certificate configured on the server must be trusted by your client. Your client must be stopped and restarted for this property to take effect.
This setting is optional and only applicable when strict SSL is enabled. If you choose to provide a custom truststore file to assert the trust of the certificate you have configured in the server, you can specify the path to your truststore file with this property.
If not set, the default truststore file provided by the your Java Runtime Environment is used. Java8 API description : A byte buffer is either direct or non-direct. Default is 0 no limit.
By default, the application will output logs every items during filtering or sorting of larger directory structure. This value may be tuned up or down as needed. Timestamp will reflect transfer time, not setup time and output an easily parsable string value containing transfer time in seconds. If this property is enabled when the MD5 is generated, the application will use an MD5 implementation that is usually much faster than standard Java-based implementations.
If this property is disabled when the MD5 is created, then a slower MD5 generation algorithm will be used to generate the MD5 checksums. The following steps illustrate how to configure so that only strong ciphers are utilized: Shutdown FileCatalyst Central: As this is a manual maconfig. Restrict the Cipher List: The following lines should be added onto the maconfig.
These can be modified to exclude less secure ciphers. By examining the logs, the complete list of supported ciphers for the platform is listed.
Please revise the allowable ciphers in the configuration. Only these specific ciphers will be used by the application server socket. If clients connecting does not have a match, the connection will be logged and dropped.
If the restrict. Compressing files can also be CPU intensive. Under most circumstances, the gain in network throughput outweighs the CPU overhead to zip up files before transfer, especially when files are text-based or other formats that can be highly compressed.
However, compression should be avoided for most binary and media files, which are normally considered non-compressible. If compression and AES on the Data Channel are utilized, high-speed transfers are likely to be limited by the amount of computing power that can be assigned for each transfer. Microsoft Windows uses different algorithms to process UDP packets based on the packet size.
While these defaults are in place, if you configure Windows to use IP standard MTU byte packets, performance is significantly degraded. For this reason, FileCatalyst software limits packet size to bytes for all Windows senders.
There is a script enableFastDatagram. For optimum performance, it is recommended to run this script, then increase packet sizes to in FileCatalyst products on Windows platforms provided the network supports standard MTU of packets. Running the script undoFastDatagram. FileCatalyst products support many tools to decrease the amount of data that is required to be sent across the network. The exact combination of features to utilize and increase file transfer performance depends on the nature of the data being sent across the network.
FileCatalyst clients allow files to be compressed before being sent out. Transferring of text files logs, emails, etc should be compressed before being transferred. It is recommended that Progressive Transfers are enabled when compression is used.
This allows the file to be sent over even as it is being compressed, reducing the overall transfer time. Media files audio, video, images however do not compress easily, and often add intensive computing overhead to a transfer while giving no bandwidth savings. Ensure that non-compressible files are filtered out by using the file compression filter.
If the same file is sent out more than once perhaps with only a small modification to the file , enabling Incremental Delta Transfers will significantly reduce the bandwidth involved in sending a file. For large files, it is also recommended to turn on Progressive Transfers. This allows the file deltas to be transferred, even as they are being built on the FileCatalyst Server, reducing the time it takes for the file to be transferred.
If the same file contains entirely new data each time example: logs truncated on daily basis, and uploaded using the same file name , it is recommended to disable incremental transfers and simply start to transfer the file as new avoid the computation required to determine what is different in the file, as all of it has changed. By default, write operations automatically flush both data and metadata to file system for all IO calls.
In situations where growing files need to be transferred, it is recommended that Progressive Transfers are enabled to improve overall performance. By enabling this option for a transfer, the FileCatalyst application transfers the file while it grows on the file system. Once the individual transfer completes, the application will monitor the files to see if they change in size within a given time threshold. Should the file change within this threshold, the modifications will be transferred, and the application will return to monitoring the file system for new changes.
The following option for Dynamic Files increases performance by consistently transferring the file contents while it is being built. By transferring the files as they grow, the amount of time that is needed for the completed file to be present on the destination is reduced. For more detailed information regarding Progressive Transfers, visit the help documentation for the FileCatalyst HotFolder application. If your data is constructed in a manner where new files are continuously being added into existing file set, it is recommended that you enable the Transfer Cache feature for your transfers.
This option reduces the amount of data being transferred, by making it so only changed or new files, are transferred by the system. Each file transfer, regardless of the file size, requires overhead to set up a network connection between the client and the server.
For large files, this overhead has minimal impact, as the software will spend most of the time transferring data across the wire.
For transfers of small individual files, however, the reverse is true: the software can spend most of the time setting up and tearing down a connection, only to transfer a directory full of many small files. To help reduce the connection overhead, there are two main ways that you can improve performance for a given transfer.
The first way to enable Single Archive Compression of small files and the second option is to use Multi-Client instead of Single-Client for transfers. By performing this operation, the application now spends all its time transferring data instead of managing setup and tear down of network connections. Enabling Progressive Transfers will also allow the application to start transferring while the single archive is being built.
Should a transfer be interrupted, auto-resume will not work, and restarting the transfer will start the archive and transfer from the beginning. This will create several smaller archives and extract them as they are transferred.
If the transfer is interrupted, generally some of the files will have already been transferred. You may also improve the performance of transferring large collections of small files by enabling the Multi-Client option available in client applications. In a normal transfer, a Single-Client may only transfer a single file at a given time, which means the application must sequentially process the connection overhead for every file transferred. Multi-Client is different in that it able to concurrently transfer files with each client that it has available.
By transferring files concurrently, the connection overhead for files is also processed concurrently across the clients, which in turn, reduces the total wait time that a transfer experiences when it is setting up its connections. Similar to the behaviour found in "Single Archive with Compression", Auto-Archiving takes small files less than 10 MB and compiles them into a zip archive with max size of MB, before it is transferred.
This option improves performance in large filesets with small file sizes by reducing the number of times the connection overhead is triggered. For clients connecting with low bandwidth, ensure that the Congestion Control start rate is set below the maximum line speed.
By default, the value for the start rate is Kbps. For very low bandwidth clients and low latency transfers, FTP mode can at times perform faster transfer rates than UDP.
Note, however, that the values in the online calculator do not take high packet loss into account. TCP at high latency is extremely sensitive to even 0. Transferring UDP at high speed is memory intensive. Using the example listed above, a ms transfer LA to Hong Kong at Kbps requires a minimum of 25 MB of data to be in-flight at any one time. This means the system can likely accept around 40 concurrent connections at full speed before starting to run short of memory resources.
The key to getting the FileCatalyst Server, to accept higher numbers of concurrent connections is to lower the memory footprint each connection can take on the system. This will lower the maximum thread count and block size each client connection consumes, resulting in the server able to accept more clients at any one time. If the maximum sender thread and maximum block size are set too low, limiting clients from connecting at full bandwidth, the following may be done:.
This allows administrators the possibility of setting the memory footprint high for each client, but limiting the connections to ensure that the FileCatalyst Server does not itself start to run short of memory. The optimizations performed for high-load servers are based upon your existing available JAVA heap size. By increasing the memory that Java has access to, the number of threads or block sizes each transfer that may use is increased, which can increase the performance of the application under certain circumstances.
For FileCatalyst HotFolders running as a service, these may be configured in fchf. Note: When using a bit verison of Java, the memory available to a Java service can be increased to 1.
You cannot currently modify the heap size in the Windows executable found in the Start Menu. For FileCatalyst HotFolders started at the command line, the starting scripts may be modified to use alternative memory sizes than the standard 1GB:. In bit versions of Java, the memory can be increased to a maximum of 4 GB Performance Tuning. Table of Contents. Performance Tuning There are many steps one can take to optimize FileCatalyst in a high speed environment.
Know Your Network Infrastructure Once the bandwidth is known, other settings can now be adjusted to accommodate your specific network environment. Other sources of Packet Loss If transfers are displaying a higher level of packet loss than your Network Packet Loss, the packet loss may be being caused by other sources. If congestion control does not correct the packet loss, the OS may not be capable of pulling the packets off of the NIC at a sufficient rate, a higher packet size may correct this Finally the process on the receiving end may not be able to keep up with the rate of packets due to lack of memory or CPU, to detect if this is a problem, setting the system property unlimited.
Compensate for High Latency To compensate for high latency, it is recommended that block sizes the amount of data each sender thread can manage at one time be increased, or that the amount of threads be increased. For typical shared land-line networks where other traffic is also present , UDP transfers should enable Congestion Control.
This allows the protocol to react to Packet Loss possibly due to other TCP traffic in a friendly manner, and slow down the rate of transfer to allow multiple traffic sources to share the line. Congestion Control Aggression value of 2 low aggressive rate enables the protocol to reacts fairly quickly.
When other traffic is detected, transfer rates should come down quickly and play nice. The start rate should also be set to be the minimum speed which you know the line is capable of handling. For dedicated land-line networks, customers often turn on the Congestion Control, but set the aggression much higher 5, or halfway , with the start rate at half the known link speed. This allows the file transfer to ramp up quickly, and maintain high rates should other traffic try to share the line.
However, if the software exceeds the link capacity, any dropped packets due to network slowdown are amplified, increasing the risk of all sender threads entering retransmission mode increases slows down transfer rate temporarily. For these types of connections, customers often turn off Congestion Control altogether, preferring to power through any temporary loss of signal and not risk slowing down the connection.
Additional strategies such as Packet Loss based Congestion Control instead of RTT based congestion control may be used for networks which drop UDP packets rather than queue them on over-saturated links. Disable Congestion Control: Transfers are sent out at prescribed speeds, with no consideration for any other network traffic on the link. Please do keep in mind that this option may push out other TCP traffic, so take care to set the bandwidth correctly.
0コメント