SnowConvert Helpers

SnowConvert Helpers is a set of classes with functions designed to facilitate the conversion of Teradata script files to Python files that Snowflake can interpret.

Mobilize.Net SnowConvert for Teradata can take in any Teradata SQL or scripts (BTEQ, FastLoad, MultiLoad and TPump) and convert them to functionally equivalent Snowflake SQL, JavaScript embedded in Snowflake SQL, and Python. Any output Python code from SnowConvert will call functions from these helper classes to complete the conversion and create a functionally equivalent output in Snowflake.

The Snowflake Connector for Python will also be called in order to connect to your Snowflake account and run the output python code created by SnowConvert.

The latest version information of the package can be found here.

The Python packagesnowconvert-helpers supports Python versions 3.6, 3.7, 3.8 and 3.9.

Script Migration

Source

Suppose you have the following BTEQ code to be migrated.

sample.bteq
insert into table1 values(1, 2);
insert into table1 values(3, 4);
insert into table1 values(5, 6);

Output

You should get an output like the one below.

The log_onfunction parameters ('user', 'password', 'account', 'database', 'warehouse', 'role') should be defined by the user.

sample_BTEQ.py
import snowconvert.helpers
from snowconvert.helpers import exec
con = None
try:
snowconvert.helpers.configure_log()
con = snowconvert.helpers.log_on('user', 'password', 'account', 'database', 'warehouse', 'role')
exec("""INSERT INTO PUBLIC.table1 VALUES (1, 2)""")
exec("""INSERT INTO PUBLIC.table1 VALUES (3, 4)""")
exec("""INSERT INTO PUBLIC.table1 VALUES (5, 6)""")
except Exception as e:
print(e)

Getting Started

To install the package, you should run the following command in your python environment. If you're not familiar with installing packages in Python, visit the following page on python packages (https://packaging.python.org/tutorials/installing-packages/).

pip install snowconvert-helpers

Once your package is installed, you will be able to run the script migrated code in Python.

Run the code

To run the migrated code, you just have to open the Command Prompt or the Terminal and execute the following command.

python sample_BTEQ.py

If the script has no errors, you will get in your console an output like the one below.

Executing: INSERT INTO PUBLIC.table1 VALUES (1, 2).
Printing Result Set:
number of rows inserted
1
Executing: INSERT INTO PUBLIC.table1 VALUES (3, 4).
Printing Result Set:
number of rows inserted
1
Executing: INSERT INTO PUBLIC.table1 VALUES (5, 6).
Printing Result Set:
number of rows inserted
1
Error Code 0
Script done >>>>>>>>>>>>>>>>>>>>
Error Code 0

Passing connection parameters

There are several ways to pass the connection parameters to the connection of the database:

  • As parameters in the function call snowconvert.helpers.log_on inside the python file.

  • As positional parameters with the specific order of user, password, account, database, warehouse and role when the python is being executed from the command line.

  • As named parameters with no order restriction of SNOW_USER, SNOW_PASSWORD, SNOW_ACCOUNT, SNOW_DATABASE, SNOW_WAREHOUSE, SNOW_ROLE and SNOW_QUERYTAG when the python is being executed from the command line and any of them are passed like --param-VARNAME=VALUE.

  • As environment variables named SNOW_USER, SNOW_PASSWORD, SNOW_ACCOUNT, SNOW_DATABASE, SNOW_WAREHOUSE, SNOW_ROLE and SNOW_QUERYTAG before python execution.

The previous order specified is the way to determine the precedence.

Parameters in the function call

They can be set as positional parameters in the function call as follows.

sample_BTEQ.py
.....
con = snowconvert.helpers.log_on(
'myuser',
'mypassword',
'myaccount',
'mydatabase',
'mywarehouse',
'myrole')
.....

Or they can be set any of the named parameters in any order in the function call as follows.

sample_BTEQ.py
.....
con = snowconvert.helpers.log_on(
account:'myaccount',
password:'mypassword',
user:'myuser',
warehouse:'mywarehouse')
.....

Positional parameters

They need to be set in the specific order in the command line as follows.

python sample_BTEQ.py myuser mypassword myaccount mydatabase mywarehouse myrole

Or they can be set only some of the parameters but always starting with the user parameter as follows.

python sample_BTEQ.py myuser mypassword myaccount

Named parameters

They can be set any of the named parameters in any order in the command line as follows (use single line, multiline shown for readability reasons).

python sample_BTEQ.py --param-SNOW_WAREHOUSE=mywarehouse --param-SNOW_ROLE=myrole
--param-SNOW_PASSWORD=mypassword --param-SNOW_USER=myuser
--param-SNOW_QUERYTAG=mytag --param-SNOW_ACCOUNT=myaccount
--param-SNOW_DATABASE=mydatabase

Environment variables

Before calling the python script must be set any of the environment variables named:

  • SNOW_USER

  • SNOW_PASSWORD

  • SNOW_ACCOUNT

  • SNOW_DATABASE

  • SNOW_WAREHOUSE

  • SNOW_ROLE

  • SNOW_QUERYTAG

Snowflake

Once any migrated code you have been executed, you can go to Snowflake and check your changes or deployments.

select * from PUBLIC.table1;

You will be able to see the rows you have inserted in the example above.

Local Helpers Documentation

First of all, it is required to install the python package named pydoc (Available since version 2.0.2 of snowconvert-helpers).

pip install pydoc

Then in order to display the python documentation of the package snowconvert-helpers, you should go to a folder where you have the converted output code and you have a python output.

D:\bteq\Output>dir
Volume in drive D is Storage
Volume Serial Number is 203C-168C
Directory of D:\bteq\Output
05/25/2021 03:55 PM <DIR> .
05/25/2021 03:55 PM <DIR> ..
05/25/2021 03:55 PM 630 input_BTEQ.py
1 File(s) 630 bytes
2 Dir(s) 1,510,686,502,912 bytes free

Located in this directory you need to run:

python -m pydoc -b

The console will open your preferred browser with the html help of the documentation for all the installed packages.

D:\bteq\Output>python -m pydoc -b
Server ready at http://localhost:61355/
Server commands: [b]rowser, [q]uit
server>

This will open the browser with the documentation of your code like:

Home page for the generated local documentation

Scroll thru the end of the page to see the installed packages. And you will see something similar to:

Local installed packages documentation index

Clicking in the snowconvert(package) you will see something like:

Home page for the snowconvert-helpers documentation

Clicking in the module helpers will display a screen similar to:

Home page for helpers module

Then you can scroll thru the functions and classes of the module.

Functions documentation