Any interest in writing software for the MPCNC?

rwillett · November 23, 2020, 8:19pm

Hi,

I’ve started looking at developing software for design and driving CNC devices and wanted to see if there was any interest in people collaborating on this.

The reason for developing new software is that none of the software currently out there is right for what I want, and possibly other people, and I think that better software can be written. Hence this post.

I have spent a lot of time thinking about this over the last few months and have had a number of attempts to work out what I think would be a viable usage design and model. I’ve put together a very simple core solution to test integration ideas out and I think it will work. However I don’t want to give people the wrong idea and that there is a finished application out there because there isn’t. There is significant work to be done, but the advantage is that you can make the software work the way you want to.

The target that I am setting for the project are as follows:

It needs to work across multiple platforms, e.g. Windows, Mac and Linux.
It needs to be open source.
It needs to be modular to allow people to extend it. I suspect that people could extend it in ways I would never imagine. However modularity is not quite as easy to do as people believe or expect
It needs to have a simple design capability built in. Minimal Viable Product (MVP) for this would be a simple 2.5D design system, e.g. simple shapes including complex concave and convex shapes with holes in them alongside depth information.
It should run locally and save work locally as well as in the cloud (which is just a term for a remote net work save).
Project files should be interchangeable between platforms.
Tooling should be simple and easy to use. If tool changes are to be done, then this should be simple to achieve on an MPCNC device.
It should interface directly to either Octoprint, CNC.js or to the latest standard offering from V1 Engineering. I am unclear what this is at the moment though
The output module should produce G-Code. The version needs to be defined and agreed. There is nothing to stop other output modules to be defined and built.
It needs to use common languages that are readily available. At this moment in time, the current version uses Electron.js for the overall wrapping of the application, fabric.js for the design panes, Angular 10 for the framework and Typescript.
The work should be stored in an easily accessible repository. Currently this is a Gitlab system hosted locally.

Work to date

So far I have built a test system to see how things fit and work. Electron.js works with fabric.js, Angular 10 and Typescript. Two minimal pages/components are setup, though they do very little, but you can move a piece of text around on a page and rotate it. ngrx is being used as a storage management layer and that’s proven complex to work with.

A Gitlab repository is setup and is available for users. It is not a pubic repository as I’m still unsure of the direction for some of the work.

There are simple build instructions that work, I’ve tested them on Linux and Mac but not on Windows, though I have no doubt it will work. Famous last words.

What needs to be done?

Basically everything. I will slowly work my way through the work but it will take time, the important thing for me is getting up to speed in Angular 10 and the new ways of working over Angular (which I know). There is a initial Kanban board that would need work to populate. I am happy for anybody to contribute but we need to be organised and methodical. I have seen too many projects fail as people get enthusiastic at the beginning and then lose the enthusiasm over time. I have no expectations of this working this year.

My plan of activity (if it was just me) would be:

Get ngrx working so there is a storage and undo/redo mechanism underpinning the application. I think it’s important to get the foundations right.
Get a simple set of shapes working, possible just a rectangle in fabric.js
Write a simple tooling selection page allowing tools to be chosen.
Write an interface class to export the data out of fabric.js with depth information to a gcode generator.
Write a simple gcode class to move the tooling around. Do not optimise at this time.
Write an interface to Octoprint as thats what I have on my CNC machine and its a public API.

Once the above is done, iterate around and then add in the nice to have things like, menu’s, file save and restore, different types of tooling paths, different shapes.

If you are interested, please let me know via message or here. If you want to know more ask, if you want to chat more, no issues, I now have so many different comms channels, from Slack, Zoom, Facetime, WhatsApp, Circuit (propriety to Atos Origin), Amazon Chimes, Skype, Skype for Business, Cisco WebEx and probably a few more I forget about.

I am more than happy to open up access to the Gitlab repository to people who are interested.

Here’'s a picture of it running on my Mac

Many thanks for reading this,

Rob

B-26 · November 23, 2020, 9:44pm

My take on the CAD software I’ve looked at is most of it is what I’d call ‘subtractive’, i.e. it has lots of capabilities but to do the relatively simple things I think many of us want to do it seems necessary to spend time sorting through everything you don’t need/want to do. Personally, I’d like to start with very basic capabilities and have the option of adding various modules as needed.

My own history, and I don’t know how prevalent/popular it was in its day, began with using MasterCam V. VIII which remains the most intuitive program I’ve ever used, I became relatively proficient simply by using it. They then introduced V. X and I was kinda lost, I’m sure it was great for those using it for hours everyday but for a hobby user like myself it was steps backwards. I understand they’re now at V. XIII but I’d love it if something like the ‘antiquated’ V. VIII became available on the hobby market.

jeffeb3 · November 24, 2020, 3:26am

That is a bit contradictory.

In the CAM I’ve seen, these are called “post processors” and they basically have some internal language (move(x,y,z, travel=true)) and then the post processor is responsible for creating the output (G0 Xx Yy Zz). Defining the language between the PP and the main CAM is the hard work, I imagine.

I didn’t see anything about importing other formats, like SVG, DXF, or STL.

I thought most electron apps ended up being packaged somehow pretty easily. Like atom, or the first slack app (they have moved away from electron now). Definitely a later stage problem.

I have a very full plate right now myself. Sounds like you have a good start on a project. I wish you luck.

rwillett · November 24, 2020, 8:15am

@B-26

My knowledge of CAM is approx nine moths old, so anybody who talks about stuff pre March 2020 has far more knowledge than me

Your point on simple is well made. I think due to the development process, I will be focussing on the simple stuff. To be honest, thats what I want as well. I need simple shapes with mostly constant cuts.

My intention with the word ‘modular’ was more to describe the underlying architecture. So somebody may add a different post processor or Gcode implementation rather than adding more user function options through modules, e.g. a light shader (just to be clear I have zero intentions of writing ray tracing software). At the moment, the dev team consists of me, so it’s what I can do in the time I have available. My focus is keep it simple and workable.

Thanks

Rob

rwillett · November 24, 2020, 8:31am

@jeffeb3

The code is open source, the repository is closed as I had no idea if I would be flooded with offers of help and I had no idea if I could manage the response. I wanted my own repository as I wanted the flexibility to set it up as I wanted to use rather than how github wants me to work. If the response is low, I’ll simply open the repository up to the public. It was an initial flow issue.

Once somebody pulls the repository, I have little control over the code and somebody else could put it up on github. Thats the way of the code. I’m not 100% certain over github since Microsoft took over. I’ve been around Microsoft’s embrace and extend philisophy for decades and I can’t say I trust MS completely. I could be wrong of course and normally am.

The CAM stuff is the way I envisage it. I can see an intermediate language produced from the design that will eventually generate Gcode or whatever. In some respects it’s a bit like the (very) old P-code system for compilers from the early 80’s. A more modern analagy could be the Android bytecode compilers. I used to love writing compilers

You are correct, I didn’t mention SVG or DXF or whatever. My assumption is SVG for initial input but that’s to be validated, so I didn’t mention it. Logically, we would have a SVG input module or a DXF input module to read the data in and convert it to whatever local format would be used for design. Currently fabric.js uses SVG as it’s internal format but it needs work to flesh this out.

Electron.js does the packaging but it’s not quite as simple as wrapping Electron.js around Angular 10. If you do that, you end up with a poor menu system and it doesn’t integrate together nicely and looks weird. Part of the integration work is to produce a proper dynamic menu system so that things like Undo and Redo are implemented correctly and all the keyboard short cuts are there. So some menu items are grayed out if not appropriate at that time. Also fabric.js captures some of the keystrokes, so you have to manage who gets what keystrokes. I got the menus working in a simple an earlier version (Aug?) but have not moved them to the latest version.

I thought that Slack still used Electron.js but they have moved more of their UI to React.js, but can’t remember where I read that.

Thanks

Rob

rwillett · December 28, 2020, 10:11am

A quick update on progress:

I now have a an app that uses Angular 10, fabric.js and Electron working on Mac and Linix. The phrase working is rather loose, it means a piece of software that starts up, stops (without crashing), and has a very, very limited functional set.
I can now import simple SVG files created and display them. I can actually import complex SVG files and display them TBH
I can move SVG elements around the screen, I can resize them and rotate them. I could easily add in simple rectangles, circles and similar but not concerned about that.
I have a menu system working that interacts between Electron and Angular and fabric.js. It only has one ite, an SVG import, but it works.
I have multiple blank pages for adding in tooling, cutting and Settings. I can move between pages and save and restore settings. That was not a trivial activity.
I have multiple TypeScript classes created for the tool. These are very simple at the moment as I’m still trying to follow TypeScript and how it actually works as opposed to JavaScript. Angular forces you to work in a certain way which is non-intuitive to me.
My focus this week is creating rulers (pet peeve of mine), that are pannable and zoomable. The classes are created but the code to make them work is still formenting in my head.
Added additional code to handle depth of each shape created. This took two weeks to get right and consisted of one line of code. Don’t you love coding …
I have a stack system for undoing and redoing worked out.

What is not done:

Tooling - descriptions of tools, how they work, sizes etc etc etc. How tools link to cutting operations. What happens when you delete a tool in a project. Some interesting points to ponder.
Settings - There are no user settings yet. Not seen as a major issue TBH.
Cutting and simulation - Mmmm… This is the big one. I have an idea how to do the cuts using a Graham Scan line algorythmn but need to think hard about it. The logic appears to be simple, you use a flood fill algorthymn with the scanline set to the diameter of the tool. I know you have to cut in certain directions depending on the rotation of the tool as well. You allso have to clean the outside of the cut up. No idea about the gcode and creation and simulation yet. Oh well, another back burner project to consider.
icons for menu bars. Ignored for moment
Error handling - Since the code will be perfect, no need. For the people who lack irony and a sense of humour, thats a joke.
Saving and restoring of projects. Not a big deal as I have the JSON routines already written.
Network connectity to cnc.js or Octoprint. Not unduely worried. This is the sort of code I used to write lots of stuff for. Never done it in JavaScript though but suspect its not that difficult.
Anything else at all

have a good new year.

Rob

rwillett · January 3, 2021, 11:45am

I thought I’d try and keep a weekly update going on progress, it gives me targets to aim for.

The focus this week was getting rulers working. This took some doing as I had to start to dig into the code in fabric.js. It also forced me to start thinking about the number of diifferent coodinate systems that will be needed. I knew this was going to a pain and it was.

The issues with the rulers are subtle and non-trivial. It’s dead easy to create a ruler with major and minor marks and put some numbering on it. What is less trivial is:

Thinking about how to display rulers in inches (spit), mm, and cm. Pixels will not be used as they are meaningless for me. View to the opposite welcomed.
How to handle rulers when you pan and zoom the main display canvas. This requires some coordination of events from the main canvas. I ended up creating four canvases for the main display, a top one for the top ruler, a left one for the left ruler, a little square at the top left called origin (more on that later) and the main canvas which is technically the bottom right. In the picture below, you can see what appears to be an off-by-one bounding box error where I draw the box on the main canvas. This is not an error but an artifact of HTML. I will remove it later.
The numbers are formatted correctly, this means that the left ruler has different formatting to the top ruler due to having to accomodate different string widths vs string heights.
Pan and Zoom are implemented on the main canvas, but I now have to propogate these events to the ruler canvases. Thats a job for next week.
I will also implement (as its easy) a vertical and horizontal scale. e.g. set it to 1.1 and you get a overall scaling of 110%. Trivial to put in as it’s needed elsewhere. Means you can draw in one unit and easily scale to another.

best wishes,

rob

rwillett · January 10, 2021, 6:24pm

Rulers have proven a lot more complex than I thought they would be. The more I looked at them, the more little niggles and issues I found.

However I think I now have a working set of rulers for the software that do what I think they need to do:

Rulers are quick to draw and use. My first iteration looked OK and I thought they were quick but the more functionality I added, the slower it got. Eventually I threw all the code away and started again. I need to read what passes for documentatin on fabric.js, but eventually I have rulers that are quick and easy to use.
I can now pan the main canvas and the rulers keep in sync. So if I’m zoomed into a design and want to pan it left and right, this now works. I can’t actually zoom yet, but neeed to get pan working correctly before doing zoom.
One thing I wanted with rulers is to have rulers with more than one cursor. Normally this is the mouse cursor so you can see the approx position in whatever units. However if I have selected a rectangle (and rotated it), when I now move the retangle it hides the mouse cursor and shows the bounding box. This was the straw that broke the camels back in point 1 and neceessitated a complete rewrite. You can see it below.

As I started to understand more about what I wanted to do, I refactored a load of code. I probably took out 50% of redundant (or more likely crap) code. This is normally a good thing

Whilst throwing away a lot of code and rewriting it sounds bad, this is a normal part of writing code. It normally means that progress is being made. I suppose I could find out from git how many lines have been deleted, but that’s a problem for another day.

Plans for this week (in addition to doing the day job, taking the dog for long walks, keeping the family happy and avoiding Covid-19) are:

Refactor more code to get rid of uncessary events.
Add in code to allow users to define the table top cutting area. This requires a modal dialogue box to be presented and as this is the first one, it needs a little thought.
Work out how to zoom the canvas and the rulers.
Write a debugging class similar to log4js that works with Electron, Typescript and Angular. All the ones I have tried (including log4js) have issues as I appear to have issues with the node_module fs not working (and it should).

Thats enough for one week.

Rob

timonjkl · January 10, 2021, 6:51pm

Wow really nice you’re moving right along

rwillett · January 11, 2021, 6:44pm

@timonjkl

Things are moving. The early parts of the project are to build the foundations for the next parts. Drawing the shapes is (relatively speaking) trivial. Getting the rulers to zoom correctly at the right part of the drawing is not trivial (at least to me).

Just to show you how trivial the drawing is, hers the very famous tiger.svg. I simply downloaded and imported it using the import function.

I can drag it and select bits of it and move them around and even scale them

(I take no credit for the drawing, thats the work of an artist).

Rob

kvcummins · January 11, 2021, 6:54pm

Have you looked at WSL at all? The biggest downside is that it takes a little more work on the user side. But it does give you a darn near full Linux façade to work from, up to and including things like an Ubuntu distro. All running on Windows 10. It’s how Docker for Windows is running on non-Enterprise Windows these days (no virtualization in the kernel).

rwillett · January 11, 2021, 7:21pm

@kvcummins

I know about the WSL subsystem but that doesn’t really help me for what I want to do. I need a common user interface for applications that works across all the major OS’s, not a subsystem for running OS level applications. WSL is great for some problems, but its not the right solution for me.

I suppose I could write the software in a Unix type system such as KDE or whatever, but I think Electron, Angular and TypeScript is a better solution for application development.

I have built a Linux box up using bits and bobs, this is my first real Linux box for around 10 years, all my Linux systems have been virtualised for the last decade or so. I develop on my Mac and test it there, and every week or so, I pull from the local repository to my Linux box to test. I also run a VMWare session on the Mac with Linux and test it there. I do have a Windows partition on my main Mac and will eventually set up a development environment on it but it’s not a pressing need. I’m not in the position of needing to fine tune it for the OS.

Next piece of work is getting Zoom working properly. I have asked for information about whether people have clicky middle wheels on their mice OR non-clicky/no wheels. It sounds a bit odd for a question but it’s really helpful for me as if most people have clicky wheels I will write code for that AND do keyboard/menu commands or if they don’t, I’ll focus on the keyboard/menu code alone. The code is not quite the same.

Anyway, thanks for the input,

Rob

Atom · January 12, 2021, 12:52pm

i don’t think it sounds odd at all, i built industrial controls for a few years and i understand what a customer has and how they are going to use it is one of the most important things when developing any user interface (and i feel like most other people on here understand that as well, but i don’t want to speak for them). i for one am impressed with the amount of effort and attention to detail you are putting into this.

KEEP UP THE GOOD WORK! and i look forward to seeing the result!

P.S.
i’m not much of programmer but if there is anything i can do to help, let me know.

rwillett · January 12, 2021, 1:04pm

Thanks. The current is to build the foundations right now. It’s a major hassle to change things like rulers, zoom and panning further downstream.

There’s all sorts of simple and not so simple decisions to be made, the biggest issue at the moment is handling rulers when you zoom into (or out of the) main drawing area. Scaling the main drawing is simple (and I mean that), it’s one or two lines of code and thats it. The rulers aren’t simple, you can’t simply scale a ruler up by zooming it, this means all the text is zoomed (yuk), all the lines are widened or lengthened and it all looks like a dogs dinner. You have to redraw the ruler with the corrrect dimensions for the zoom level, but that takes time, not long, but approx 10the of a second on my box. If you quickly zoom through 10 levels, thats a second of time just to redraw all the rulers on the top AND another second to draw the rulers on the left. Two seconds is too long just for rulers, so I have to make it faster and slicker, this probably means caching.

It’s all interdependent and everything is a compomise to get speed and efficiency.

Anyway, enough of my whining back to the day job.

Thanks

Rob

Atom · January 12, 2021, 1:10pm

i know your no where near this yet, but maybe you have a document or something where you are keeping track of possible features to add later. i think it would be great if you could add milling for joint cuts. there is another program out there that costs over $100, but it would be nice to have some basic functionality in a more reasonably priced program.

rwillett · January 12, 2021, 1:25pm

I’d like to get milling for single cuts in first

Rob

rwillett · January 20, 2021, 7:51pm

Progress on the software has moved forward. The big issues this week has beenL

Getting zooming on a canvas, both in and out at the mouse position. This is a lot harder than it sounds as it needs the rulers which are on two separate HTML canvas’s syncronised as the main canvas zooms in and out. This means the ruler scale has to change and be readable. I have used the a and z keys for the moment as the mouse wheel, my prefered option is quite a lot of work.
Getting panning working. This is where the canvas is moved around.Currently this is by mouse button and the middle key on my keyboard. No idea how this works on Linux and Windows as haven’t tested it. As with zooming, you have to syncronise all the HTML canvases.
Clicking on the top left corners immediately moves all the rulers to (0,0). Whilst this is kinda nice, the real reason for this work is that you need an event bus in place to do it as you have to pass events up to the main canvas from non connected canvases and Angular 10 is an utter nightmare as it no longer supports broadcast events. The event bus will eventaullly be used for undo’s and redo’s and other stuff.
The cache system is now in place and works well.
The toolbar has been revamped to use SVG icons instead of fonts. This means that I (or somebody with design skills) can make the SVG for me. As they are SVG they are highly scalable which might be nice. To test the tool bar works, I can now create rectangles on the main canvas by clicking on it. I also added in code to make the rulers visible and invisible from the toolbar.
I can now delete multiple objects such as rectangles from the main canvas.

I would show a video of this but don’t think I can put a video here.

Next bits of work planned is still around the backend. As I walk thedog each morning, I plan my working day and work on my code as bits of time become available.

The big focus now is likely to be the gcode generation. This requires:

A standard way to export the data used to hold the shape. This includes all the appropriate tooling information such as which bits are used for cutting. I have zero idea about tools apart from the diameter. What is the impact of double fluted bits? What is trapzeoidal cutting? Do I care?
This interface design has to be flexible and robust as my intention is that people could write their own cutting routines based on a standard interface. The transport mechanism will be http(s) using a REST API, local pipes (where supported) and clearly internallly. I want the interface to decumented as to east my own dog foof when using it.

I have a rough idea the process to generate gcode from shapes but need to do a load of work to get it working so this might be a few months of work.

Hope this gives people an idea of whats going on.

All the best

Rob

jeffeb3 · January 20, 2021, 9:49pm

I would have thought this would all just be replacing a transform matrix you apply to each coordinate before drawing. But maybe that is the hard or expensive way, since each vertex would have to be redrawn.

The material design icons are pretty complete and are great for stuff like this.

Gifs are great for screen grabs.

rwillett · January 21, 2021, 6:10am

@jeffeb3

Viewports only provide part of the solution, specifically the x or y offset. You can’t scale the numbers or lines in a ruler the same way as the main image. The numbers and stroke widths of the major and minor lines have to be consistent so each ruler is redrawn. I can’t have the ruler line widths double if we 2x zoom.

Other considerations are that the numbers need to change, when zoomed in, we are working with sub mm numbers, when zoomed out, it goes to 2400mm so you can’t display every number.

There is also a speed issue, as you zoom and n and out, you can’t completely draw the ruler and main image fast enough for complex images before the screen refreshes. So I have to cache each ruler. That works pretty well. There’s a lot of vectors for each ruler.

The Material images look OK. However I did see a deal on icon54 images for $19 so brought that as they had loads of appropriate icons.

Used GIF Brewery to create the GIF.

jeffeb3 · January 21, 2021, 4:28pm

Gif looks great.