Anatomy of SuiteScript: Map/Reduce And Scheduled Scripts

Introduction

Scheduled scripts and map/reduce scripts both accomplish a similar task: they run a script either on a schedule or when manually triggered. Scheduled scripts have just a single entry point: execute. Map/reduce scripts have multiple entry points as we’ll see below. Map/reduce scripts, as the name implies, run a script on an input array that you provide in one of the entry points.

While nothing is preventing you from using a scheduled script to iterate over an array and run a script on each function, there are two reasons why you might want to use a map/reduce script instead:

• You might hit the script usage limit. See the NetSuite documentation for more information.
• Map/reduce scripts can run processes in parallel for elements in the array resulting in a faster execution time.

Let’s first look at a scheduled script as it is much simpler than a map/reduce script.

Scheduled Script

Here is a scheduled script generated by the vscode extension:

/**
 * @NApiVersion 2.1
 * @NScriptType ScheduledScript
 */
define([],

    () => {

        /**
         * Defines the Scheduled script trigger point.
         * @param {Object} scriptContext
         * @param {string} scriptContext.type - Script execution context. Use values from the scriptContext.InvocationType enum.
         * @since 2015.2
         */
        const execute = (scriptContext) => {

        }

        return {execute}

    });

That’s it. That’s all there is to a scheduled script. Load whatever modules you need and fill in the execute function.

Map/Reduce Script

Map/reduce scripts are a bit more complicated.

Here is a map/reduce script generated by the vscode extension:

Click to expand

/**
 * @NApiVersion 2.1
 * @NScriptType MapReduceScript
 */
define([],

    () => {
        /**
         * Defines the function that is executed at the beginning of the map/reduce process and generates the input data.
         * @param {Object} inputContext
         * @param {boolean} inputContext.isRestarted - Indicates whether the current invocation of this function is the first
         *     invocation (if true, the current invocation is not the first invocation and this function has been restarted)
         * @param {Object} inputContext.ObjectRef - Object that references the input data
         * @typedef {Object} ObjectRef
         * @property {string|number} ObjectRef.id - Internal ID of the record instance that contains the input data
         * @property {string} ObjectRef.type - Type of the record instance that contains the input data
         * @returns {Array|Object|Search|ObjectRef|File|Query} The input data to use in the map/reduce process
         * @since 2015.2
         */

        const getInputData = (inputContext) => {
        }

        /**
         * Defines the function that is executed when the map entry point is triggered. This entry point is triggered automatically
         * when the associated getInputData stage is complete. This function is applied to each key-value pair in the provided
         * context.
         * @param {Object} mapContext - Data collection containing the key-value pairs to process in the map stage. This parameter
         *     is provided automatically based on the results of the getInputData stage.
         * @param {Iterator} mapContext.errors - Serialized errors that were thrown during previous attempts to execute the map
         *     function on the current key-value pair
         * @param {number} mapContext.executionNo - Number of times the map function has been executed on the current key-value
         *     pair
         * @param {boolean} mapContext.isRestarted - Indicates whether the current invocation of this function is the first
         *     invocation (if true, the current invocation is not the first invocation and this function has been restarted)
         * @param {string} mapContext.key - Key to be processed during the map stage
         * @param {string} mapContext.value - Value to be processed during the map stage
         * @since 2015.2
         */

        const map = (mapContext) => {
        }

        /**
         * Defines the function that is executed when the reduce entry point is triggered. This entry point is triggered
         * automatically when the associated map stage is complete. This function is applied to each group in the provided context.
         * @param {Object} reduceContext - Data collection containing the groups to process in the reduce stage. This parameter is
         *     provided automatically based on the results of the map stage.
         * @param {Iterator} reduceContext.errors - Serialized errors that were thrown during previous attempts to execute the
         *     reduce function on the current group
         * @param {number} reduceContext.executionNo - Number of times the reduce function has been executed on the current group
         * @param {boolean} reduceContext.isRestarted - Indicates whether the current invocation of this function is the first
         *     invocation (if true, the current invocation is not the first invocation and this function has been restarted)
         * @param {string} reduceContext.key - Key to be processed during the reduce stage
         * @param {List<String>} reduceContext.values - All values associated with a unique key that was passed to the reduce stage
         *     for processing
         * @since 2015.2
         */
        const reduce = (reduceContext) => {

        }


        /**
         * Defines the function that is executed when the summarize entry point is triggered. This entry point is triggered
         * automatically when the associated reduce stage is complete. This function is applied to the entire result set.
         * @param {Object} summaryContext - Statistics about the execution of a map/reduce script
         * @param {number} summaryContext.concurrency - Maximum concurrency number when executing parallel tasks for the map/reduce
         *     script
         * @param {Date} summaryContext.dateCreated - The date and time when the map/reduce script began running
         * @param {boolean} summaryContext.isRestarted - Indicates whether the current invocation of this function is the first
         *     invocation (if true, the current invocation is not the first invocation and this function has been restarted)
         * @param {Iterator} summaryContext.output - Serialized keys and values that were saved as output during the reduce stage
         * @param {number} summaryContext.seconds - Total seconds elapsed when running the map/reduce script
         * @param {number} summaryContext.usage - Total number of governance usage units consumed when running the map/reduce
         *     script
         * @param {number} summaryContext.yields - Total number of yields when running the map/reduce script
         * @param {Object} summaryContext.inputSummary - Statistics about the input stage
         * @param {Object} summaryContext.mapSummary - Statistics about the map stage
         * @param {Object} summaryContext.reduceSummary - Statistics about the reduce stage
         * @since 2015.2
         */
        const summarize = (summaryContext) => {

        }

        return {getInputData, map, reduce, summarize}

    });

Life Cycle

Four entry points are defined in a map/reduce script:

• getInputData
• map
• reduce
• summarize

The general life cycle of a map/reduce script is:

1. getInputData is called. Do whatever logic you need - saved search, suitelet, etc. - and return the input data.
2. map is called for each key/value pair in the input data. This stage should be used to transform the keys to group them to be used in the reduce stage.
3. shuffle is called. This is not an entry point and is not defined in the script. NetSuite will group key/value pairs if the keys match from the pairs returned by the map stage.
4. reduce is called for each of the key/value pairs. This is where you perform the logic that you need to on each key/value pair. This is where you can use write data to be used in the summarize function.
5. summarize is called. This is where you can use data from the reduce function if you have some cleanup or final tasks to do. This is only called once.

Of these entry points, only getInputData is strictly required. Only define summarize if you need to do some final cleanup or processing of the data. Between map and reduce one of them must be defined. Nevertheless I recommend always defining the redice function and using the map function as needed. Let’s take a look at when you might need the map stage.

When to Use the Map Stage

To understand the map and reduce stages, let’s imagine I have 4 work orders for 2 customers:

Customer Id	Work Order Id
Customer 1	Work Order 1
Customer 1	Work Order 2
Customer 2	Work Order 3
Customer 2	Work Order 4

There’s two ways I might want to process this data. Either I want to process each work order separately or I might want to process all of the work orders for customer 1 together and all of the work orders for customer 2 together.

If I want to process each work order separately, I can omit the map stage and just return the work order ids in the getInputData function. In the reduce stage I can process each work order individually.

It is possible to use the map stage to process each work order individually, but because the reduce stage is needed if you want to write data to be used in the summarize stage or if you want to group the data, I recommend always using the reduce stage.

What if I want to process all of the work orders for customer 1 together and all of the work orders for customer 2 together? Then I will need to use the map stage to transform the keys somehow so that NetSuite will group them together when it calls the reduce stage.

When I call getInputData I will have it return an array that just contains the work order ids.

const getInputData = (inputContext) => {
    return [
        'Work Order 1',
        'Work Order 2',
        'Work Order 3',
        'Work Order 4',
    ]
}

Then I will use the map stage to transform the key to be the customer id.

const map = (mapContext) => {
    const workOrderId = mapContext.value
    const customerid = getCustomerid(workOrderId)
    return {
      [customerid]: workOrderId
    } /* { 'Customer 1': 'Work Order 1' } */
}

Now after the map stage, before the reduce stage, NetSuite will group the work orders by customer id.

There are the four values returned by the map stage:

[
    { 'Customer 1': 'Work Order 1' },
    { 'Customer 1': 'Work Order 2' },
    { 'Customer 2': 'Work Order 3' },
    { 'Customer 2': 'Work Order 4' },
]

NetSuite will group the values by their keys, in this case the customer id:

  [
      { 'Customer 1': [ 'Work Order 1', 'Work Order 2'] },
      { 'Customer 2': [ 'Work Order 3', 'Work Order 4'] },
  ]

Now NetSuite will pass each group of work orders separately in the reduce stage.

Entry Points

getInputData

The getInputData function is called when the map/reduce script is triggered.

getInputData can return many different types of data. Three common structures are:

• A simple array
• An array of objects
• An object

I will use Input to refer to the data returned by getInputData and Output to refer to the data passed to the map stage (each element being another call to the reduce stage).

If you return an array, NetSuite will consider the index of the array to be the key and the value to be the value.

Input: ['Work Order 1', 'Work Order 2', 'Work Order 3', 'Work Order 4']

Output: {0: 'Work Order 1', 1: 'Work Order 2', 2: 'Work Order 3', 3: 'Work Order 4'}

If I return an array of objects, NetSuite will consider the value of the id property to be the index and the value to be a JSON string of the object.

Input: [ { id: 'Work Order 1', name: 'Work Order 1' }, { id: 'Work Order 2', name: 'Work Order 2' }]

Output: { '0': '{"id":"Work Order 1","name":"Work Order 1"}', '1': '{"id":"Work Order 2","name":"Work Order 2"}' }

If I return an object, NetSuite simply uses the keys and values of the object.

Input: { 'Work Order 1': '123', 'Work Order 2': '324' }

Output { 'Work Order 1': '123', 'Work Order 2': '324' }

You can also do more exotic things like return a query object or a file object. I have found that it is much more practical to either load the file or run the query in the getInputData function and return the results manually.

map

The context object passed to the map function contains the following properties we care about:

• mapContext.key - The key of the key/value pair
• mapContext.value - The value of the key/value pair

We process the data however we see fit and return an object with the key/value pair we want to use in the reduce stage. In this function we are only returning one key/value pair, as such it should simply be an object with a single property.

reduce

Now we finally get to the stage where we process our data. The context object passed to the reduce function contains a few properties:

• reduceContext.key - The key of the key/value pair
• reduceContext.values - An array of values for all of the key/value pairs with the same key returned by the map function
• reduceContext.write - A function that can be used to write data to be used in the summarize function

An example of using the reduceContext.write function:

const reduce = (reduceContext) => {
    const customerid = reduceContext.key
    const workOrderIds = reduceContext.values
    const total = getTotal(customerid, workOrderIds)
    reduceContext.write({
        customerid,
        total,
    })
}

summarize

The summarize function is called after all of the reduce functions have been called.

The context object passed to the summarize function contains many properties related to the script execution such as script usage, concurrency, etc.

In addition the summaryContext object contains three objects related to the other stages:

• summaryContext.inputSummary - An object containing information about the data returned by the getInputData function
• summaryContext.mapSummary - An object containing information about the data returned by the map function
• summaryContext.reduceSummary - An object containing information about the data returned by the reduce function

For more information on the summaryContext object, see the NetSuite documentation.

To get the data we saved in the reduce function, we use the summaryContext.output.iterator() function.

const summarize = (summaryContext) => {

  summaryContext.output.iterator().each((key, value) => {
    const { customerid, total } = JSON.parse(value)
    log.debug('customerid', customerid)
    log.debug('total', total)
    return true
  })
}

Conclusion

A full example of a map/reduce script will be coming soon.

Map/reduce scripts take a while to get the hang of. Although for smaller tasks you may be able to get away with a scheduled script, inevitably you will run into a situation where you are running out of script usage units or it is taking too long to run. In these situations, map/reduce scripts are a great solution.

In this blog I strive to make this content as accessible as possible, and as such there is a lot of information that I have left out. The NetSuite documentation is really where you should head after understanding the basics.

Starting to write simple map/reduce scripts without a map stage or without a summarize stage is a great way to get started.

Feel free to leave any questions or comments below.