Builtins

This chapter describes the AXIS namespace when executing custom scripts in a scenario or custom methods in ATOM.

In the namespace, there are a python module imported by default, an AXIS built-in function, an atom module, a model module, and a context containing plug-in modules and functions.

Module

python module imported by default into the namespace is as follows:

asyncio datetime ipaddress json re sqlalchemy statistics time traceback urllib uuid xmltodict yaml

When a developer installs additional modules by pip etc., the default import is performed by importing directly in the script or specifying module names separated by commas in --python_default_imports in the startup parameter of AXIS.

--python_default_imports=boto3,kubernetes

Function

category	name	type	description
Basic	Error	class	Exception class. It is strongly recommended to use this class when throwing an exception.
	HTTPError	class	Exception class. It may become unusable in future releases due to deprecation.
	options	module	Module used to reference startup parameters.
	logger	object	Logger object.
	context	object	Object with accessor to plug-in module or function.
API	callout	function	API call function.
	status_poll	function	API polling function. It can be used to monitor state transitions.
	queryjoin	function	Function to convert dictionary type to query string and generate URL.
asyncIO	multi	function	Function to wait for multiple coroutines. Same behavior as asyncio.gather.
	Awaitable	function	Function to co-routineize synchronous blocking call. It is recommended to use this function for co-routineization.
	futureization	function	Function to co-routineize synchronous blocking call. It may be discontinued in the future because of deprecation.
	add_timeout	function	Function to delay function execution.
Schedule	regist_booking	function	Reservation registration function to schedule service.
	cancel_booking	function	Reservation cancellation function to schedule service.
	modify_booking	function	Reservation modification function to schedule service.
	retrieve_booking	function	Reservation search function from schedule service.
Transaction	get_transactions	function	Function to retrieve transaction from transaction service.
	transaction_active_check	function	Function to check if the specified transaction is still running.
Lambda	lambda_event	function	Function that raises lambda event.
	lambda_terminate	function	Function to wait for completion of lambda event and get the result.
	lambda_bulk_terminate	function	Function to wait for completion of multiple lambda events and get list result in list.
Scenario	Scenario.load	function	Function to get scenario and return in execution format.
	Daemon.load	function	Function to get daemon and return in execution format.
	Scenario.run	function	Function to run the specified scenario.
	Daemon.run	function	Function to run the specified daemon.
	get_service_config	function	A function to get the config with the specified name defined in the Config service.
	rendering	function	A function to get the rendering result of the template service to the specified template.
	task	function	Function to issue task
RDBMS	model	module	For object mapping of the table created from modelschema and ATOM and DB connection creation Context Manager is included in the package.
	and_	function	AND conditional expression function (see SQL Alchemy)
	or_	function	OR conditional expression function (see SQL Alchemy)
	not_	function	NOT conditional expression function (see SQL Alchemy)
	select	function	SELECT expression function (see SQL Alchemy)
	join	function	JOIN expression function (see SQL Alchemy)
	where_statement	function	Function to generate WHERE statement
	rowtodict	function	Function to convert row fetched from database to dict of python
	SQL.run	function	Query function
GraphDB	neo4j	function	Function to get session to Neo4j
	nodetodict	function	Function to convert node fetched from graph database to dict of python
Redis	Redis	class	Context manager for manipulating redis. Use by plug-in developers is deprecated.
	pubsub	function	Publish Function to notify. Use by plug-in developers is deprecated.
ATOM	deserialize	function	Function to restore JSON string to ATOM instance.
	drop_class	function	Function to delete ATOM class. Use by plug-in developers is deprecated.
	clear_redis_pool	function	Function to clean the redis connection pool. Use by plug-in developers is deprecated.
XML	dict2xml	function	Function to convert a dictionary object to an XML string.
	ElementTree	class	Accessor to ElementTree class of xml library.
gRPC	gRPC	function	Function to get the gRPC client.
	Parse	function	Protocol buffer parser.
	MessageToJson	function	Function to convert gRPC response to json string.
SNMP	SNMPv2	class	SNMPv2 client
	SNMPv3	class	SNMPv3 client
Cluster	raft	object	An object to access cluster information in Raft cluster mode.
Utilities	magic	function	A function that returns dict with the key filter and key name conversion rule specified fordict.
	MU	class	It is Mutable Universal object. See the chapter on Datamodel.
	Frozen	class	It is a class that makes dict a frozen object. This class is deprecated, please use MU.
	clock	object	Utility object used for time processing.
	ipam	object	Accessor object to IPAM service.
	scp	function	Secure Copy function.
	slack	function	Function to send a message to slack. You need to set the startup parameters appropriately.
	genkey	function	Password generation function.
	encrypt	function	Encryption function.
	decrypt	function	Decryption function.
	flattening	function	Function that converts a multi-level key dict object into a one-level dict object. The second and subsequent layers are stringified.

Interactive shell

Interactive Shell is a feature that allows plug-in developers to try out the scripting of AXIS like REPL. The interactive shell works with the AXIS script execution namespace, so you can write and test code snipet with the above built-in and plug-ins.

Let's use some built-in functions in the interactive shell.

Example of using queryjoin

>>> print(queryjoin("/hoges", status=["active", "processing"], name="test"))↵
... ↵
/hoges?status=active&status=processing&name=test↵
↵

Example of using multi

>>> responses = await multi([callout(url="http://yahoo.co.jp") for i in range(10)])↵
... print([i.code for i in responses])↵
... ↵
[200, 200, 200, 200, 200, 200, 200, 200, 200, 200]↵
↵

Example of using Awaitable

>>> print(clock.now())↵
... await Awaitable(time.sleep, 3)↵
... print(clock.now())↵
... ↵
2019-04-24 11:11:39.392098+09:00↵
↵
2019-04-24 11:11:42.395468+09:00↵
↵

Example of using genkey

>>> print(genkey())↵
... ↵
D5AFlqE4yUoGPfUVGYuMJmm4RRli8jqk86Y3pkWfxUjYbsX3n9FyAzF8jqfHXEEJ↵
↵
>>> print(genkey(length=10, continuity=2))↵
... ↵
u66dkDHVab↵
↵`

Supported SQL execution from v19.11R1

...Connecting To : Scenario Master(192.168.172.141:9099)↵
AXIS Wild Side v19.11R1 [core-dev-py37(192.168.172.141)]
2019-11-06T10:10:53.859171+09:00
Type "copyright", "help", "commands", "tree"
↵
>>> a = atom.Aaa(name="hoge1", b=atom.Bbb(name="hoge2"))↵
... await a.save()↵
... ↵
... select * from Aaa;↵
| instance | xid | xname | name | b | status |
QWFhOjUzZWI0MjYwMDAzMjExZWE4MGIxMDAwYzI5ZGQ4MjUw NULL NULL hoge1 QmJiOjUzZWIyYzU4MDAzMjExZWE4MGIxMDAwYzI5ZGQ4MjUw NULL↵
>>> update Aaa set name="hoge123" where name="hoge1";↵
>>> select * from Aaa;↵
| instance | xid | xname | name | b | status |
QWFhOjUzZWI0MjYwMDAzMjExZWE4MGIxMDAwYzI5ZGQ4MjUw NULL NULL hoge123 QmJiOjUzZWIyYzU4MDAzMjExZWE4MGIxMDAwYzI5ZGQ4MjUw NULL↵
>>> a = await atom.Aaa.load("QWFhOjUzZWI0MjYwMDAzMjExZWE4MGIxMDAwYzI5ZGQ4MjUw")↵
... print(a.name)↵
... ↵
hoge123↵
>>> desc Aaa;↵
| Field | Type | Null | Key | Default | Extra |
instance varchar(255) NO PRI NULL 
xid varchar(255) YES  NULL 
xname varchar(255) YES  NULL 
name varchar(255) YES UNI NULL 
b mediumtext YES  NULL 
status varchar(255) YES  NULL ↵
>>>

Example of using SNMPv2

...Connecting To : 127.0.0.1:9099(127.0.0.1:9099)
Qmonus-SDK v20.7LTS-patch20201015 [UG30-hash-dev.usen.ad.jp(192.168.2.103)]
2020-10-19T11:21:48.273723+09:00
Type "copyright", "help", "commands", "tree"
>>> snmp = SNMPv2("public")↵
... r = await snmp.get("192.168.2.200", oid="1.3.6.1.2.1.1.5.0")↵
... print(MU(r).yaml_format)↵
... ↵
address: 192.168.2.200
errorIndex: null
errorIndication: null
errorStatus: null
errorVarBind: null
objectType: DisplayString
oid: 1.3.6.1.2.1.1.5.0
value: csr1000v.ftth.ucom.ne.jp
↵
>>>
>>> r = await snmp.bulk("192.168.2.200", oids=["1.3.6.1.2.1.1"])↵
... for i in r:↵
...     print(MU(i).yaml_format)↵
... ↵
address: 192.168.2.200
errorIndex: null
errorIndication: null
errorStatus: null
errorVarBind: null
oid: 1.3.6.1.2.1.1.1.0
type: DisplayString
value: "Cisco IOS Software, CSR1000V Software (X86_64_LINUX_IOSD-UNIVERSALK9-M), Version\
  \ 15.5(2)S, RELEASE SOFTWARE (fc3)\r\nTechnical Support: http://www.cisco.com/techsupport\r\
  \nCopyright (c) 1986-2015 by Cisco Systems, Inc.\r\nCompiled Sun 22-Mar-15 01:36\
  \ by mcpre"
address: 192.168.2.200
errorIndex: null
errorIndication: null
errorStatus: null
errorVarBind: null
oid: 1.3.6.1.2.1.1.2.0
type: ObjectIdentity
value: SNMPv2-SMI::enterprises.9.1.1537
...
>>>
>>>
>>> r = await snmp.getInterfaces("192.168.2.200")↵
>>> for i in r:↵
...    print(i)↵
... ↵
{'index': '1', 'description': 'GigabitEthernet1'}
{'index': '2', 'description': 'GigabitEthernet2'}
{'index': '3', 'description': 'GigabitEthernet3'}
{'index': '4', 'description': 'VoIP-Null0'}
{'index': '5', 'description': 'Null0'}
↵
>>>

HTTP functions

callout

API calls use the callout built-in function. The main arguments are as follows:

argument	default	description
url	None	When specified, send directly from the scenario server to the specified URL (APIGW does not pass)
endpoint	None	Send to specified endpoint. It is invalid when URL is specified.
path	None	Specifies the HTTP request path.
method	None	Specifies the HTTP request method. Default is GET.(POST|PUT|DELETE|GET|PATCH)
body	None	Specifies the HTTP request body.(str|list|dict types)
static_route	None	Set the value in X-Xaas-Static-Routing of the request header.
connect_timeout	None	Specifies the HTTP connect timeout. The default is the startup parameter --callout_connect_timeout.
request_timeout	None	Specifies the HTTP request timeout. The default is the startup parameter --callout_request_timeout.
retry_interval	None	Specify retry interval when HTTP error code specified in retry_codes is received.
retry_count	None	Specify retry count when HTTP error code specified in retry_codes is received.
retry_codes	None	Specifies the HTTP error code to be retried.
authentication	None	Specifies authentication options.
lambda_termination	None	Specifies whether to wait for the end of the lambda event and return the result in the case of the lambda proxy API call.
validate_cert	None	For HTTPS requests, validate the server’s certificate? Default is True.
ca_certs	None	Filename of CA certificates in PEM format, or None to use defaults.
client_key	None	Filename for client SSL key, if any.
client_cert	None	Filename for client SSL certificate, if any.

Example of use

Direct communication from scenario server to endpoint

response = await callout(url="https://www.yahoo.co.jp")

Communication via APIGW

response = await callout(path="/v2.0/tenants")

BASIC certification

response = await callout(path="/example/basicAuthentications",
                         authentication=dict(auth_mode="basic",
                                             username="admin",
                                             password="admin"))

DIGEST certification

response = await callout(path="/example/digestAuthentications",
                         authentication=dict(auth_mode="digest",
                                             username="admin",
                                             password="admin"))

Keystone certification

response = await callout(path="/v2.0/tenants",
                         authentication=dict(auth_mode="keystone",
                                             username="{username}",
                                             password="{password}",
                                             project_id="{project_id"}))

AWS-HMAC-V4 certification

response = await callout(url="directconnect.ap-northeast-1.amazonaws.com",
                         method="POST",
                         body=dict(connectionId="xxxxx"),
                         authentication=dict(auth_mode="AWS-HMAC-V4",
                                             username="xxxxx",
                                             password="xxxxx",
                                             domain_id="ap-notrheast-1",
                                             domain_name="DirectConnect",
                                             project_id="DescribeVirtualInterfaces"))

Retry in case of exclusive error (409 Conflict) and communication disconnection (599)

response = await callout(path="/v2.0/tenants",
                         method="POST",
                         body=dict(name="Sample tenant"),
                         retry_count=2,
                         retry_interval=3,
                         retry_codes=[409, 599])

status_poll

In resource control with REST-API, there are many cases of status polling. The status_poll built-in function can write status polling simply.

argument	default	description
url	None	When specified, send directly from the scenario server to the specified URL (APIGW does not pass)
endpoint	None	Send to specified endpoint. It is invalid when URL is specified.
path	None	Specifies the HTTP request path.
method	None	Specifies the HTTP request method. Default is GET.(POST|PUT|DELETE|GET|PATCH)
body	None	Specifies the HTTP request body.(str|list|dict types)
connect_timeout	None	Specifies the HTTP connect timeout. The default is the startup parameter --callout_connect_timeout.
request_timeout	None	Specifies the HTTP request timeout. The default is the startup parameter --callout_request_timeout.
retry_interval	None	Specify retry interval
retry_count	None	Specify retry count
authentication	None	Specifies authentication options.
xid	None	Transactions can be aborted during state polling. If you specify xid, transaction confirmation is performed at polling time, and abort will stop it.
status_key	None	Key name to monitor as status attribute
expected_status	[]	Expected state value candidate. If it matches, it will return a tuple of True and the response object.
unexpected_status	[]	Not expected state value candidate. If it matches, it will return a tuple of False and the response object.
break_codes	[]	HTTP response code candidate leaving status polling. If it matches, it will return a tuple of True and the response object.
break_schemas	[]	HTTP body schema candidate leaving status polling. If it matches, it will return a tuple of True and the response object.
fault_codes	[]	Failure judgment HTTP response code candidate. If it matches, it will return a tuple of False and the response object.
countdown_hook	None	Specify a function to hook status polling.

Example of use

result, response = await status_poll(path="/v2.0/networks/72a1b7a667e011e993ac000c29dd8250",
                                     retry_interval=5,
                                     retry_count=60,
                                     xid=context.axis.xid,
                                     status_key="status",
                                     expected_status=["active"],
                                     unexpected_status=["error"])

Database

Persistence tables automatically generated by modelschema or ATOM are ORM by SQLAlchemy and dynamically incorporated under the model module. A function for nonblocking execution of the connection to the database is also provided in the model module.

The RDBMS supported by AXIS are MySQL, PostgreSQL and SQLite3. The built-in functions and module used to operate the database are as follows:

name	type	description
model	module	For object mapping of the table created from modelschema and ATOM and DB connection creation Context Manager is included in the package.
and_	function	AND conditional expression function (see SQL Alchemy)
or_	function	OR conditional expression function (see SQL Alchemy)
not_	function	NOT conditional expression function (see SQL Alchemy)
select	function	SELECT expression function (see SQL Alchemy)
join	function	JOIN expression function (see SQL Alchemy)
where_statement	function	Function to generate WHERE statement
rowtodict	function	Function to convert row fetched from database to dict of python
SQL	function	Query function
neo4j	function	Function to get session to Neo4j
nodetodict	function	Function to convert node fetched from graph database to dict of python

Database connections should use the async with syntax as follows:

async with model.aiodb() as conn:
    # Inside async with you can build and execute SQL freely with the syntax of SQLAlchemy.
    cursor = await conn.execute(model.employee.select())
    employees = await cursor.fetchall()

Database transaction is as follows:

async with model.aiodb() as conn:
    async with conn.begin() as tran:
        cursor = await conn.execute(model.employee.select().where(model.employee.c.status=="retired"))
        employees = await cursor.fetchall()
        [await conn.execute(model.employee.delete().where(model.employee.c.id==i.id)) for i in employees]

Alternatively you can use the SQL built-in function

# SQLAlchemy
employees = await SQL(model.employee.select())

# Raw SQL statement
employees = await SQL("SELECT * FROM employee;")

# When using SQL function in transaction scope
async with model.aiodb() as conn:
    async with conn.begin() as tran:
        employees = await SQL(model.employee.select(), conn=conn)

How to use where_statement

employees = await SQL(model.employee.select().where(model.employee.where_statement({"status": "retired"})))

How to use neo4j and nodetodict

>>> with neo4j() as sess:↵
...     r = sess.run("MATCH (n)\nRETURN n")↵
...     for i in r:↵
...         print(nodetodict(i))↵
... ↵
{'name': 'top1', 'instance': 'VG9wOjhlMzg4OWM0N2I3NTExZTliMDBhMDAwYzI5ZGQ4MjUw', 'age': 10, 'status': True}↵
↵