Graphics Programming

NeuroKernel comes with an extensive graphics library. Graphics is one of the strong features of NeuroKernel API. It is effectively available even when the application run at the server side. It has a optimized system protocol that can cache drawings at the display server for performance. Below example shows how a display list is used to cache the drawing. It is possible to turn a plain NComponent into a drawable canvas similar to Swing although a more specialzed NCanvas class is available for the purpose. A graphics context either 2D or 3D must be created to start drawing.

1. 2D Graphics

NCanvas class is the preferred way of drawing graphics. It is possible to give an initial drawing to the canvas that can be cached as seen in the below example. Canvas class has features to utilize offscreen canvas as well if available. It can also double buffer drawing.

import com.neurokernel.client.*;
import com.neurokernel.client.graphics.*;
 
public class CanvasExample extends NApplication {
    @Override
    public void main(int argc, String[] argv) {
       NFrame mainFrame=getMainFrame();
 
       NCanvas canvas=new NCanvas(mainFrame);
       IGraphics2D ctx = canvas.getGraphics();
       ctx.setFillStyle(NColor.BLUE);
       ctx.rectangle(10, 10, 55, 50);
       ctx.fill();
       ctx.setFillStyle(NColor.RED);
       ctx.rectangle(30, 30, 55, 50);
       ctx.fill();
 
       mainFrame.setTitle("Canvas Example");
       mainFrame.setBounds(20,20,200,200);
       mainFrame.setVisible(true);
    }
}

2. 3D Graphics

3D graphics interface used in NeuroKernel API uses WebGL graphics context available at the client side. It is recommended to use excessive amount of GPU layers which will slow down the system allocation a huge memory chunk for the operations.

import com.neurokernel.client.*;
import com.neurokernel.client.graphics.*;
 
public class CanvasExample extends NApplication {
    @Override
    public void main(int argc, String[] argv) {
        NFrame mainFrame=getMainFrame();
        new N3DGraphics(mainFrame);
 
        mainFrame.setTitle("Canvas Example");
        mainFrame.setBounds(20,20,200,200);
        mainFrame.setVisible(true);
    }
 
    static class N3DGraphics extends NCanvas {
        private IGLProgram program;
 
        public N3DGraphics(NContainer con) {
            super(con, GraphicsContext.CONTEXT_3D);
        }
 
        private void initGraphics(IGraphics3D gl) {
            String vs = "attribute vec2 pos; void main() { gl_Position = vec4(pos, 0, 1); }";
            String fs = "precision mediump float; void main() { gl_FragColor = vec4(1,0,0,1); }";
            program = gl.createProgram(vs, fs);
            gl.linkProgram(program);
            gl.useProgram(program);
            IGLBuffer vertexPosBuffer = gl.createBuffer();
            gl.bindBuffer(NGL.ARRAY_BUFFER, vertexPosBuffer);
        }
 
        @Override
        public void paint(IGraphics graphics) {
            IGraphics3D gl = (IGraphics3D) graphics;
            if (gl == null) return;
            if (!isPainted()) initGraphics(gl);
            double[] vertices = {-0.5, -0.5, 0.5, -0.5, 0, 0.5};
            gl.bufferData(NGL.ARRAY_BUFFER, vertices, NGL.STATIC_DRAW);
            gl.bindAttribLocation(program, 0, "pos");
            gl.clearColor(1, 1, 1, 1);
            gl.clear(NGL.COLOR_BUFFER_BIT);
            gl.enableVertexAttribArray(0);
            gl.vertexAttribPointer(0, 2, NGL.FLOAT, false, 0, 0);
            gl.drawArrays(NGL.TRIANGLES, 0, 3);
        }
    }
}

3. Display Lists

In order to optimize graphics performance, display lists are used to cache drawing protocol in task context at the client side. Both 2D and 3D graphics can be cached in a display list. The most notable advantage of this can be seen when the application runs at the server or at a peer computer.

import com.neurokernel.client.*;
import com.neurokernel.client.graphics.*;
 
public class GraphicsExample extends NApplication {
   @Override
   public void main(int argc, String[] argv) {
      NFrame mainFrame=getMainFrame();
 
      NComponent canvas = new NComponent(mainFrame) {
          IDisplayList mygraphics;
 
           @Override
           public void paint(IGraphics graphics) {
               IGraphics2D context = (IGraphics2D) graphics;
               if(mygraphics==null) {
                   mygraphics=context.newList();
                   context.setStrokeStyle(new NColor(128, 128, 255));
                   context.setFillStyle(new NColor(0, 0, 255));
                   context.setLineWidth(5);
                   double pi = 2 * Math.PI;
                   context.beginPath();
                   context.arc(50, 50, 100, 0, pi, false);
                   context.closePath();
                   context.fill();
 
                   context.beginPath();
                   context.arc(50, 50, 100, 0, pi, false);
                   context.closePath();
                   context.stroke();
                   context.endList();
               }
               context.translate(100, 100);
               context.callList(mygraphics);
           }
       };
 
       mainFrame.setTitle("Drawing Graphics");
       mainFrame.setBounds(20,20,200,200);
       mainFrame.setVisible(true);
    }
}

4. Graphics Context Association

Using NPlugin component, it is possible to associate a graphics context(GC) with the plugin application using associateCanvas method. The plugin task will be alerted about the association, and it can draw on the canvas using graphics protocol. Plugin application can also create a Canvas component and may ask its graphics context from task container to directly manipulate the canvas if it is a client side task. If task is a remote task or a worker task, this feature requires that the browser used has offscreen canvas capabilities.

   import com.neurokernel.client.*;
   import com.neurokernel.client.system.IPluginSystem;
 
   public class MyApplication extends NApplication {
      @Override
      public void main(int argc, String[] argv) {
          NFrame mainFrame=getMainFrame();
 
          NCanvas canvas=new NCanvas(mainFrame);
          NPlugin plugin=new NPlugin(mainFrame, "teapot", null);
          IPluginSystem pluginSystem=plugin.getPluginSystem();
          pluginSystem.associateCanvas("MyCanvas", canvas.getGraphics());
 
          mainFrame.setTitle("Plugin Application");
          mainFrame.setBounds(20,20,200,200);
          mainFrame.setVisible(true);
       }
    }
 

5. Native Graphics Context

Native panel is available when application is run at the client side. In server side runtime, it acts as a plain graphics canvas. Native pane can be used to create visual experiences using third party libraries. It also has the ability to hook NeuroKernel graphics interface to a canvas element which makes direct manipulation possible without going through protocol layer. Example below shows the use of a HTML canvas element as a IGraphics interface.

import com.neurokernel.client.*;
import com.neurokernel.client.graphics.*;
 
public class NativePaneExample extends NApplication {
    @Override
    public void main(int argc, String[] argv) {
       NFrame mainFrame=getMainFrame();
       NNativePane nativePanel = createNativePane(mainFrame);
       IGraphics2D ctx = nativePanel.getGraphics();
       ctx.setFillStyle(NColor.BLUE);
       ctx.rectangle(10, 10, 55, 50);
       ctx.fill();
       ctx.setFillStyle(NColor.RED);
       ctx.rectangle(30, 30, 55, 50);
       ctx.fill();
 
       mainFrame.setTitle("Native Pane");
       mainFrame.setBounds(20,20,200,200);
       mainFrame.setVisible(true);
    }
}