oomnitza/oomnitza-connector

* Python2 Deprecation Notice:

Since version 2.1.0 the connector is migrated to the python3.6.x (or above) codebase.

The further maintenance, support and development of the python2 based version is NOT planned.

Please make sure you have converted ALL custom converters and filters to py3 syntax before upgrading the connector to the 2.1.0 or above.

Local connector

Oomnitza’s local connector, built using Python, is a single application that pulls data from multiple vendor applications and pushes data into your Oomnitza instance.

The local connector can pull data from the following sources:

• Google Chrome devices https://developers.google.com/admin-sdk/directory/
• Chef https://www.chef.io/chef/
• Jasper [***]
• KACE Systems Management Appliance [***]
• LDAP e.g., http://www.openldap.org/, Active Directory
• MobileIron [***]
• Netbox https://netbox.readthedocs.io/en/stable/
• Open-AudIT [***]
• SCCM [***]
• SimpleMDM [***]
• Tanium [***]
• vCenter https://www.vmware.com
• WorkspaceOne [***]
• Plain CSV files

Before you begin

The local connector can be used to create two types of integrations:

• Basic integrations
• Extended integrations

Basic integrations

Basic integrations run on our local connector and can presently pull data from the sources listed above. To create a basic integration, you complete these actions:

• Download the code for the basic integration from GitHub
• Update the configuration file
• Push the configuration file to your Oomnitza instance
• Map the vendor application’s fields to Oomnitza

Extended integrations

!integrationtypes_basic_and_extended

The initial setup and configuration of extended integrations that use the local connector is similar to that of basic integrations. However, when an extended integration is pushed to your Oomnitza instance, you can avail of additional features in the Oomnitza UI to:

• Create and manage schedules
• Review detailed error logs
• Easily add mapping fields

To set up an extended integration locally, complete the steps and set the connector to run in managed mode. Some of the reasons why customers choose to install extended integrations locally are as follows:

• To access systems that cannot be accessed from the Oomnitza Cloud
• To store credentials locally. That is, you don’t want to store connection credentials in the Oomnitza Cloud.
• To connect to systems such as Microsoft Endpoint Configuration Manager (MECM), Lightweight Directory Access Protocol (LDAP), and VMWare VCenter.

Alternatively, you can create extended connector integrations that run in the Oomnitza Cloud. When you run extended integrations in the Oomnitza cloud, you get more more benefits and features such as:

• The onboarding wizard to set up instances in minutes.
• The Oomnitza vault with built-in security for storing credentials in the Cloud. See Adding credentials to the Oomnitza vault.
• More authentication options such as OAuth and AWS.
• The syncing of SaaS users.

We also have more extended cloud integrations to choose from! To view a complete list of our supported extended integrations navigate to Configuration>Extended>New Integration in your Oomnitza instance or checkout our documentation.

If you can’t find an integration, you can request one. See how to request extended integrations.

• Getting Started
• System Requirements
• Runtime Environment Setup
  • Linux Environment
  • Windows Environment
  • OS X Environment
• Containerized Environment Setup
  • Overview
  • Before you start
    • Download the GitHub repository
    • Install the GitHub repository
  • Download and install Docker Desktop
  • Run the local connector with Docker Compose
    • Initial configuration
    • Modify the config.ini file
    • Run the local connector
  • Add service examples
    • LDAP service
    • CSV Assets service
• Connector Configs
  • Common optional settings
  • Oomnitza Configuration
• Storage for Connector secrets
  • Common recommendations
  • Deployment and receiving secrets
    • Local KeyRing
    • HashiCorp Vault
    • CyberArk secret storage
      • Self-hosted CyberArk installation
      • Managing secrets via CyberArk
      • Connector configuration
• Running the connector server
• Running the connector client
  • Setting the connector to run in managed mode
    • SaaS authorization item
    • Oomnitza authorization item
    • Local inputs item
    • Setting the export file connector
  • Setting the connector to run in upload mode
    • Setting the connector to run as an automated task for upload mode
    • CSV Assets Configuration
    • CSV Users Configuration
    • Chef Configuration
    • Google Chrome Devices
    • Jasper Configuration
    • KACE SMA Configuration
    • LDAP Users Configuration
    • LDAP Assets Configuration
    • MobileIron Configuration
    • Netbox Configuration
    • Open-AudIT Configuration
    • SCCM Configuration
    • SimpleMDM Configuration
    • Tanium Configuration
    • vCenter Configuration
    • WorkspaceOne Configuration
• Advanced usage
  • Logging
  • Custom Converters
  • Record Filtering
• Current limitations
  • Software mapping
  • MS Windows environment

Getting Started

The most current version of this documentation can always be found on https://github.com/Oomnitza/oomnitza-connector/blob/master/README.md.

Use this local connector to run basic integrations and to run extended integrations locally.

To run basic integrations, follow the steps to install and configure the local connector.

To run extended integrations locally, follow the steps to install the local connector and set the connector to run in managed mode.

If you would prefer to run extended integrations in the Oomnitza Cloud, refer to extended integrations.

System Requirements

The Oomnitza Connector supports Linux, Windows, and Mac OS. For Linux, we recommend the Ubuntu OS. Recommended Requirements: 2-4 vCPU, 4-8 GB RAM, 2 GB disk space. Overall the Connector has a small footprint and only utilizes CPU and RAM during the scheduled synchronization jobs. With log rotation enabled for the generated Connector logs, the consumed disk space will remain within limits. Depending on the number of integrations and volume of data, the Connector can be configured to use additional workers.

Runtime Environment Setup

You will need to install Python 3.6.X (or above) as well as the packages which the connector relies upon. Some of the python packages may require build tools to be installed.

Please visit the sections below related to the build tools before installing the additional modules.

We suggest you setup a virtual environment and use pip to install the requirements. This can be done as follows (See our documentation on installing additional Python modules for use in Oomnitza.):

cd /path/to/connector
virtualenv .
source ***/activate
pip install --upgrade pip
pip install -r requirements.txt 

Linux Environment

On Ubuntu, the build tools are installed using:

sudo apt-get install build-essential unixodbc unixodbc-dev

Windows Environment

For MS Windows you have to install Windows C++ compilers as build tools. Please visit the https://wiki.python.org/moin/WindowsCompilers To check what is appropriate compiler you have to download and install.

OS X Environment

For OS X environment you have to install the build tools using the following command:

xcode-select --install

Containerized Environment Setup

Overview

Before we begin installing the local connector, it is important to think ahead about what data you want to bring in and how you want to store it in Oomnitza. Since Oomnitza is highly customizable, there are many possibilities. Before proceeding to the next steps, take time to think about what information you want, and what Oomnitza fields you want filled out with data. Complete the following steps in your Oomnitza instance:

• Click Configuration > Integrations.
• In the Basic section for Asset or User Integrations, click the tile corresponding to the integration, such as Workspace ONE.
• In the Mappings section, map the integration fields to the Oomnitza fields.
• Select a unique identifier for the sync key, which will synchronize the data that is streamed from your integration to Oomnitza.

If the fields you want to map to Oomnitza haven’t been created yet, refer to our Guide to creating custom fields in Oomnitza.

Before you start

To run the local connector in a docker container, you must:

• Download and install the GitHub repository
• Download and install Docker Desktop

Download the GitHub repository

• Go to the https://github.com/Oomnitza/oomnitza-connector#connector-configs page on GitHub.
• Scroll to the top of the page, click Code, and then click Download Zip.

Install the GitHub repository

You create a directory to download and install the GitHub repository on a local drive.

For example, in Windows, you create a directory called myconfig in this file path: C:\oomnitza\connector\myconfig

In Linux, you create a directory called myconfig in this file path: /home/myconfig

Download and install Docker Desktop

Click a link to download Docker Desktop:

• https://www.docker.com/products/docker-desktop
• https://hub.docker.com/search?offering=community&operating_system=linux&q=&type=edition
• https://hub.docker.com/editions/community/docker-ce-desktop-mac?utm_source=docker&utm_medium=webreferral&utm_campaign=dd-smartbutton&utm_location=header

Run the local connector with Docker Compose

Initial configuration

To start the local connector using Docker Compose, you must complete these steps:

1. Save the template for the connector configuration file, docker-compose.yml, in the directory, for example myconfig, that you created.

2. Open the docker-compose.yml file and replace /path/on/local/machine in the volumes section with the path on your local machine.

   Windows: Replace /path/on/local/machine with C:\oomnitza\connector\myconfig.

   Linux: Replace /path/on/local/machine with /home/myconfig.

3. Open a command line in the directory that you created, for example myconfig, and issue the following command:

   docker-compose up generate-ini

Result

The generated configuration file (config.ini) is copied to the directory on your local machine.

Modify the config.ini file

To set up the local connector for your basic or extended integration, and to connect the local connector to your Oomnitza instance, you must modify the config.ini file. The config.ini file tells the local connector which Oomnitza Cloud instance to connect to and which basic or extended integration the local connector should serve up to the Oomnitza Cloud instance.

If you intend to run the local connector solely to connect to systems that are behind a firewall, you only need to maintain the [oomnitza section] and one or more of the managed sections in the config.ini file.

For more information, see https://github.com/Oomnitza/oomnitza-connector/blob/master/README.md#setting-the-connector-to-run-in-managed-mode.

Run the local connector

To run the local connector, issue the following command:

docker-compose up oomnitza-connector -d

Result

The docker container will run in detached mode. That is, as a background process.

Add service examples

If you need to run extended integrations, you can add an additional service to the Docker Compose configuration file, docker-compose.yml.

LDAP service

For LDAP, you add:

oomnitza-connector-ldap:
  image: oomnitza/oomnitza-connector:latest
  command: python connector.py --ini ../config/config.ini upload ldap
  volumes:
    - /path/on/local/machine:/home/appuser/config/

CSV Assets service

For CSV assets, you add:

oomnitza-connector-csv-assets:
  image: oomnitza/oomnitza-connector:latest
  command: python connector.py --ini ../config/config.ini upload csv_assets --testmode
  volumes:
    - /path/on/local/machine:/home/appuser/config/
    - /another/path/on/local/machine:/home/appuser/exp/

The CSV file that contains the asset records should be stored in a directory on the local machine, the path in the container should be defined in the configuration file. For example, /home/appuser/exp/<file_name>.csv.

Example

[csv_assets]
enable = True
sync_field = BARCODE
filename = /home/appuser/exp/assets.csv
directory = 
mapping.BARCODE = {"source": "Barcode"}

Important

If you run Docker on a Windows 10 desktop, you might need to enclose the Windows folder path with single or double quotes in the volumes section.

Connector Configs

Now you should be able to generate a default config file. Running python connector.py generate-ini will regenerate the config.ini file, and create a backup if the file already exists. When you edit this file, it will have one section per connection. You can safely remove the section for the connections you will not be using to keep the file small and manageable.

If you require multiple different configurations of a single connector, such as the need to pull from two different LDAP OUs, additional sections can be added by appending a '.' and a unique identifier to the section name. For example, having both a [ldap] and [ldap.Contractors] section will allow you to pull users from a default and Contractor OU.

An example generated config.ini follows.

[oomnitza]
url = [***]
api_token = 
username = oomnitza-sa
password = ThePassword

[managed.xxx]
enable = False
saas_authorization = {"params": {"api-token": "saas-api-token"}, "headers": {"Authorization": "Bearer Example"}}
oomnitza_authorization = oomnitza-api-token
local_inputs = {"username": "***", "password": "ThePassword"}
test_run = false

[chef]
enable = False
url = [***]
client = user
key_file = /path/to/user.pem
attribute_extension = 

[chromebooks]
enable = False
service_account_impersonate = ***
service_account_json_key = {}

[csv_assets]
enable = False
filename = /some/path/to/file/assets.csv
directory = /some/path/to/files/
sync_field = 24DCF85294E411E38A52066B556BA4EE

[csv_users]
enable = False
filename = /some/path/to/file/users.csv
directory = /some/path/to/files/
default_role = 25
default_position = Employee
sync_field = USER

[jasper]
enable = False
wsdl_path = [***]
username = ***
password = change-me
api_token = YOUR Jasper API TOKEN
storage = storage.db

[kace]
enable = False
url = [***]
username = ***
password = ***
organization_name = Default
api_version = 8

[ldap]
enable = False
url = ldaps://ldap.com:389
username = cn=read-only-admin,dc=example,dc=com
password = 
base_dn = dc=example,dc=com
group_dn = 
protocol_version = 3
filter = (objectClass=*)
default_role = 25
default_position = Employee
page_criterium = 
groups_dn = []
group_members_attr = member
group_member_filter = 

[ldap_assets]
enable = False
url = ldaps://ldap.com:389
username = cn=read-only-admin,dc=example,dc=com
password = 
base_dn = dc=example,dc=com
group_dn = 
protocol_version = 3
filter = (objectClass=*)
page_criterium = 
groups_dn = []
group_members_attr = member
group_member_filter = 
sync_field = 24DCF85294E411E38A52066B556BA4EE

[mobileiron]
enable = False
url = [***]
username = ***
password = change-me
partitions = ["Drivers"]
api_version = 1
include_checkin_devices_only = True
last_checkin_date_threshold = ***

[netbox]
enable = False
url = [***]
auth_token = *******

[open_audit]
enable = False
url = [***]
username = 
password = 

[sccm]
enable = False
server = server.example.com
database = CM_DCT
username = change-me
password = change-me
authentication = SQL Server
driver = 

[simplemdm]
enable = False
secret_access_key = ***
device_groups = 
device_types = computers,mobiledevices
custom_attributes = 0

[tanium]
enable = False
url = [***]
username = ***
password = ***
domain = 
view = 

[vcenter]
enabled = False
url = [***]
username = ***
password = change-me

[workspaceone_devicesoftware]
enable = False
subdomain = tech-dev
client_id = ***
client_secret = ***
region = na

The [oomnitza] section is where you configure the connector with the URL and login credentials for connecting to Oomnitza. You can use an existing user’s credentials for username and password, but best practice is to create a service account using your standard naming convention. See the documentation for managing user accounts in Oomnitza.

The remaining sections each deal with a single connection to an external service. The "enable" field is common to all connections and if set to "True" will enable this service for processing. Some fields are common to a type of connection. For example, "default_role" and "default_user" are fields for connections dealing with loading People into the Oomnitza app.

Each section can end with a list of field mappings. Simple mappings which just copy a field from the external system to a field inside Oomnitza can be defined here or in the System Settings within Oomnitza. Simple mappings are as follows:

mapping.[Oomnitza Field] = {"source": "[external field]"}

For fields which require processing before being brought into Oomnitza must be defined in the INI. These mappings are more involved. Please contact *** for more information. The format is:

mapping.[Oomnitza Field] = {"source": "[external field]", "converter": "[converter name]"}

Common optional settings

sync_field: The Oomnitza field (fields) which contains the object's unique identifier. We typically recommend username or email for users and serial_number for assets. Will be loaded from Oomnitza mapping if not set. To create multiple sync field, split it by comma, for example sync_field = USER,EMAIL. The exceptions for this rule are the LDAP assets & CSV files because there is no way to set the mapping in Oomnitza for these data sources at the current moment.

insert_only: set this to True to only create records in Oomnitza. Records for existing objects will not be updated.

update_only: set this to True to only update records in Oomnitza. Records for new objects will not be created.

insert_only and update_only can not be both of true value.

verify_ssl: set to false if the target data source instance is running with a self signed or invalid SSL certificate.

ssl_protocol : if the service to be connected to requires a particular SSL protocol version to properly connect, the connection's section in the ini file can include a ssl_protocol option. The value can be one of: ssl, sslv23, sslv3, tls, tls1.

Oomnitza Configuration

url: the url of the Oomnitza application. For example: https://example.oomnitza.com

username: the Oomnitza username to use

password: the Oomnitza password to use

api_token: The API Token belonging to the Oomnitza user. If provided, username and password will not be used. For further information, refer to Creating an API token.

user_pem_file: The path to the PEM-encoded certificate containing the both private and public keys of the user. Has to be used only if there is enabled two factor authentication in your environment. The certificate has to be also uploaded to Oomnitza in the "Configuration/ Security/ Certificates" page.

Storage for Connector secrets

To prevent secrets sprawl and disclosure the Oomnitza Connector uses secret backends to securely store credentials, usernames, API tokens, and passwords.

There are three options:

• local KeyRing;
• external Vault Key Management System by Hashicorp (the Vault KMS);
• external CyberArk Secrets Management

KeyRing (KeyChain) is a secure encrypted database and the easiest to configure.

The https://www.vaultproject.io/intro/index.html and CyberArk provide an additional layer of security. In this case, all secrets will be stored in the external encrypted system

Common recommendations

Before adding secrets for Connector, first, follow the instructions and setup the Oomnitza Connector. Use a technical role with restricted permissions to run the Connector.

De