Configuration¶

Lime Admin¶

With CSV Import, it is possible to configure multiple jobs. Configuration below is done for each job.

Job Configuration¶

Job settings:

Name	Default value	Description
Unique job name	`N/A`	Used as reference when scheduling job.
Integration monitor key	`N/A`	Used to map batches and logs to the integration monitor object. See configuration in Lime CRM.
Job is active	`False`	Set to True if job should be active.
Import source	`SFTP`	SFTP is currently the only supported import source.
Import files	`N/A`	Files to import. File settings are given for each file.

File settings:

Import file config

Name	Default value	Description
Unique import name	`N/A`	Used as reference for file import.
Inactive	`False`	Set to True if import job for this specific file should not be executed.
File name	`N/A`	Name of the file to import.
Delimiter	`;`	Delimiter used in the file.
Limetype	`N/A`	Table to create/update objects in.
Run settings	`N/A`	Run settings for each file import. See below.
Lookup keys	`N/A`	List of lookup keys used to search for existing objects in Lime CRM. See below.
Field mapping	`N/A`	Field mapping from external system to Lime CRM. See below.

Run settings:

Name	Default value	Description
Create	`N/A`	Create new objects if not found in Lime.
Update	`N/A`	Update existing objects in Lime.
Commit in batches	`N/A`	Commit creates/updates in batches. Better for performance, but can create duplicates in Lime if there exists duplicates in the import file.
Commit batch size	`100`	Number of object to commit in each batch.

Lookup keys:

This package supports lookup with a combination of properties and an order of lookup keys. For example: Search for person with email adress = 'jane.doe@test.se', if not not found search for person with firstname = 'Jane' and lastname = 'Doe'. If found - update person 'Jane Doe', otherwise create a new Object.

Field mapping:

Name	Default value	Description
External field	`N/A`	Field in external system.
Lime property	`N/A`	Lime property. If relation, choose key property on related table.
Option mapping	`N/A`	Only visible if Lime property is Option field.
Date format	iso format	Format to parse the value with. If nothing is specified, `datetime.fromisoformat` is used.
Do not overwrite	`False`	Do not overwrite existing value in Lime.

Integration Monitor Administrators / Users¶

Make sure that correct users are added to Groups Integration Monitor Administrators and Integration Monitor Users. See Integration Monitor Documentation.

Application configuration¶

CSV Import uses application configuration to store settings for SFTP server for each job configured in Lime Admin. '' below is replaced with unique job name configured in Lime admin.

Windows on-premise Installations¶

For Windows on-premise installations, the application configuration is expressed in the file:

%ProgramData%\Lundalogik\LIME Pro Server\application_config.yaml

Add following to the application_config.yaml file:

<application-name>:
  config:
    limepkg-csv-import:
      <unique-job-name>:
        sftp:
          address: ""
          port: ""
          username: ""
          remote_location: ""
          done_folder: ""
          error_folder: ""

  secrets:
    limepkg-csv-import:
      <unique-job-name>:
        sftp:
          password: ""

Name	Description	Example value
address	address to customers sftp	ftp.customer.com
port	sftp port (default is 22)	22
username	username to sftp	ftp_user
password	password to sftp	super_secret
remote_location	directory on sftp that contains the files	Lime
done_folder	folder name for done files	done
error_folder	folder name for error files	error

Cloud¶

For Cloud installations, the application configuration is configured in Cloud admin.

Add following to the Configuration box for your application in Cloud admin:

{
  "limepkg-csv-import": {
    "<unique-job-name>": {
      "sftp": {
        "address": "",
        "port": "",
        "username": "",
        "remote_location": "",
        "done_folder": "",
        "error_folder": ""
      }
    }
  }
}

Name	Description	Example value
address	address to customers sftp	ftp.customer.com
port	sftp port (default is 22)	22
username	username to sftp	ftp_user
remote_location	directory on sftp that contains the files	Lime
done_folder	folder name for done files	done
error_folder	folder name for error files	error

Add following to the Secret box in the application configuration in CAFE:

{
  "limepkg-csv-import": {
    "<unique-job-name>": {
      "sftp": {
        "password": ""
      }
    }
  }
}

Name	Description	Example value
password	password to sftp	super_secret

Lime CRM¶

Create Integration Monitor object in Lime CRM with integration_key given in job configuration in Lime Admin.

Schedule job¶

Scheduling each import job is done in your customer's solution. It is best practice to use a dedicated integration user for each job. Make sure that the user has Create/Read/Update access to all tables included in your configuration.

If you don't have tasks in your solution, generate one (lime-project generate tasks) and just delete the generated task-file, so you just have the init file left. The tests/tasks-folder can also be removed if you don't have tasks of your own.

The schedule can be anything you want, just make sure that the external system exports files with the same interval.

Scheduling a job can look something like this in your tasks/init.py file:

from lime_task.schedule import CronTab, ScheduledTask, TaskSession
from limepkg_csv_import.tasks import run_import_job as limepkg_csv_import


def get_task_modules():
    return []


def register_scheduled_tasks():
    return [
        ScheduledTask(
            task=limepkg_csv_import.run_import_job,
            session=TaskSession(user_name="task@external_system_import", language="sv"),
            args=["external_system"],  # Unique job name given in Lime Admin
            schedule=CronTab(minute="0", hour="0"),
        ),
        ScheduledTask(
            task=limepkg_csv_import.run_import_job,
            session=TaskSession(user_name="task@external_system_2_import", language="sv"),
            args=["external_system_2"],  # Unique job name given in Lime Admin
            schedule=CronTab(minute="0", hour="8-17"),
        ),
    ]