Interfacing the Poplar SDK

A quick example of use of the Poplar SDK functionalities, available in the IPUToolkit.Poplar submodule:

julia> using IPUToolkit.Poplar

julia> dm = Poplar.DeviceManager();

julia> Int(Poplar.DeviceManagerGetNumDevices(dm))
129

julia> device = Poplar.get_ipu_device();
[ Info: Trying to attach to device 0...
[ Info: Successfully attached to device 0

julia> Int(Poplar.DeviceGetId(device))
0

julia> Poplar.detach_devices()

A couple of basic examples of programs running on the IPU written using the interface to the Poplar SDK are available in the files examples/tutorial1.jl and examples/tutorial2.jl.

We automatically generate the bindings of the Poplar SDK using Clang.jl and CxxWrap.jl. There is not automatic documentation at the moment, but functions can be accessed from the Poplar submodule. Also, the IPUToolkit.Poplar submodule wraps a subset of the functionalities available in the Poplar SDK, do not expect to be able to use all functionalities. Remember that Julia does not use class-based object-oriented programming, class instances will usually be first arguments of the methods you want to use.

Function naming convention and signature is usually as follows:

class name in CamelCase, followed by method name also in CamelCase. Note that first letter of method name is always uppercase in this naming convention, even if it is lowercase in the Poplar SDK. For example, the method getNumDevices of the DeviceManager class can be accessed in the Poplar submodule with Poplar.DeviceManagerGetNumDevices;
the first argument of the function is the class instance. For example, to use the Julia function Poplar.DeviceManagerGetNumDevices, you need to pass as first argument an instance of DeviceManager;
the following arguments are the same as in the method you want to use in the SDK. For example, the method getNumDevices of the DeviceManager class doesn't take any argument, so the Julia function Poplar.DeviceManagerGetNumDevices will take an instance of DeviceManager as only argument.

Convenient methods

In addition to this, for some functions (e.g. EngineWriteTensor, EngineConnectStream, EngineReadTensor) we provide more user-friendly methods where the last argument can be a Julia's Array, without having to pass additional arguments for pointers or array size. Furthermore, the custom functions Poplar.get_ipu_device and Poplar.get_ipu_devices can be used to access one more IPU devices, as shown in the example above.

Another function for which we provide a convenient method is Poplar.GraphAddConstant:

Poplar.GraphAddConstant(graph, host_array)

adds the host_array (a plain standard Julia Array living on the host) to graph, automatically inferring from host_array the type and the shape of the tensor in the graph. This works also with host_array::Array{Float16}.

You can slice a tensor with the usual Julia notation tensor[index1:index2], this corresponds to a call to Tensor.slice(index1, index2+1).

similar can be used to add to graph a tensor with the same shape and optionally element type as tensor, while copyto! can be used to copy elements of a CPU host array into an IPU tensor.

Using `IPUToolkit.jl` without an IPU

While this package requires a physical IPU to use all the available features, you can still experiment with the IPU programming model even if you do not have access to a hardware IPU. The Poplar SDK provides a feature called IPU Model, which is a software emulation of the behaviour of the IPU hardware. While the IPU model comes with some limitations, it can be useful for testing or debugging.

To use the IPU model in IPUToolkit.jl, define the device of your IPU program with Poplar.get_ipu_model:

device = Poplar.get_ipu_model()
# Then the rest of the program continues as usual
target = Poplar.DeviceGetTarget(device)
graph = Poplar.Graph(target)
# ...

Base.copyto! — Method

copyto!(
    graph::Poplar.Graph,
    dest::Poplar.TensorAllocated,
    src::Array
) -> Poplar.TensorAllocated

In the given graph copies the elements of the CPU host array src to the IPU tensor dest, using Graph::setInitialValue under the hood. The elements of src must have a type corresponding to the type of dest (e.g. Float16 for a half IPU tensor, or Float32 for a float IPU tensor). This function returns dest.

source

Base.similar — Method

similar(
    graph::Poplar.Graph,
    tensor::Union{Poplar.TensorAllocated,Array},
    [type::DataType],
    [debug::String]
) -> Poplar.TensorAllocated

Adds to graph a variable tensor with the same shape as tensor, which can be either an IPU tensor or a plain CPU host Array, using Graph::addVariable under the hood. If a type (this is a Julia type, like Float32 or Int32) argument is not passed, the same element type as tensor will be automatically used. An optional debug context can also be passed, as a String. This function returns a pointer to the tensor added to the graph.

source

IPUToolkit.Poplar.detach_devices — Method

Poplar.detach_devices() -> Nothing

Detach all devices previously attached in the current Julia session with Poplar.get_ipu_devices, Poplar.get_ipu_device, or Poplar.get_ipu_model.

source

IPUToolkit.Poplar.get_ipu_device — Function

Poplar.get_ipu_device(hint::Union{AbstractVector{<:Integer},Integer}=0) -> Poplar.DeviceAllocated

Similar to Poplar.get_ipu_devices, but request exactly one IPU device. If it can attach to a device, return that pointer only (not in a vector, like get_ipu_devices), otherwise return nothing.

See Poplar.get_ipu_model for requesting an IPU Model.

You can release the device with Poplar.DeviceDetach(device). To release all devices previously attached with Poplar.get_ipu_device, Poplar.get_ipu_devices, or Poplar.get_ipu_model use Poplar.detach_devices.

The optional argument hint suggests to which device IDs to try and attach. It can have different types:

if of type Integer, try to attach to one device, starting from the one with index hint. The default is hint=0;
if of type AbstractVector, try to attach to a device from that list of IDs.

source

IPUToolkit.Poplar.get_ipu_devices — Function

Poplar.get_ipu_devices(n::Int, hint::Union{AbstractVector{<:Integer},Integer}=0) -> Vector{Poplar.DeviceAllocated}

Try to attach to n IPU devices, returns a vector of the pointers to the devices successfully attached to. You can release them with Poplar.DeviceDetach (note that this function takes a single pointer as input, so you have to use broadcasting Poplar.DeviceDetach.(devices) to release a vector of pointers).

The second optional argument hint suggests to which device IDs to try and attach. It can have different types:

if of type Integer, try to attach to n devices, starting from the one with index hint. The default is hint=0;
if of type AbstractVector, try to attach to n devices from that list of IDs.

See Poplar.get_ipu_device for requesting exactly one IPU device, and Poplar.get_ipu_model for requesting an IPU Model. To release all devices previously attached with Poplar.get_ipu_devices, Poplar.get_ipu_device, or Poplar.get_ipu_model use Poplar.detach_devices.

source

IPUToolkit.Poplar.get_ipu_model — Function

Poplar.get_ipu_model(ipu_version::String="ipu2") -> Poplar.DeviceAllocated

Attach to an IPU Model, and return the attached device. This uses IPUModel::createDevice under the hood.

The optional positional argument ipu_version::String, ipu2 by default, represents the version of the IPU to emulate. Valid values for ipu_version are ipu1 and ipu2 (for Mk1 and Mk2 IPU architectures respectively).

See Poplar.get_ipu_device and Poplar.get_ipu_devices for requesting one or mode hardware IPUs.

You can release the device with Poplar.DeviceDetach(device). To release all devices previously attached with Poplar.get_ipu_model, Poplar.get_ipu_device or Poplar.get_ipu_devices use Poplar.detach_devices.

source

IPUToolkit.Poplar.@graph — Macro

@graph [graph] expr

This is a convenient macro to automatically inject graph as first argument of all function calls in the expression passed as last argument to the macro.

The graph argument should be the graph object you want to pass as first argument to the function calls. If it is a local variable called exactly graph, this argument can be omitted and this name will be used automatically.

Note

This macro is not very sophisticated and will fail with complex expressions involving, for example, control flows like if or for. See the examples below.

Examples

julia> @macroexpand @graph begin
           c1 = Poplar.GraphAddConstant(Float32[1.0, 1.5, 2.0, 2.5])
           v1 = similar(c1, "v1")
           copyto!(v1, Float32[4.0, 3.0, 2.0, 1.0])

           Poplar.GraphSetTileMapping(c1, 0)
           Poplar.GraphSetTileMapping(v1, 0)
       end
quote
    c1 = Poplar.GraphAddConstant(graph, Float32[1.0, 1.5, 2.0, 2.5])
    v1 = similar(graph, c1, "v1")
    copyto!(graph, v1, Float32[4.0, 3.0, 2.0, 1.0])
    Poplar.GraphSetTileMapping(graph, c1, 0)
    Poplar.GraphSetTileMapping(graph, v1, 0)
end

source

Interfacing the Poplar SDK

Convenient methods

Using IPUToolkit.jl without an IPU

Using `IPUToolkit.jl` without an IPU