Firmware (Marlin) build and release process for V1 machines

This is awesome. I think this is really going to help us and especially Ryan.

I need to poke it some more, and I’m especially interested in how much work building for 2.0.1 is going to be. Maybe we should add a version or build number in case we change the configs. Something like 2.0.1.0?

1 Like

If some1 wants to play around with it, this is how you start:

git clone https://github.com/anttix/Marlin.git playground
cd playground
git submodule update --init --remote

# This will fetch Marlin's bugfix-2.0.x merge in a few sample PR-s, merge some branches (e.g. v1-base-config), generates configs and stores it in "build" branch
v1-scripts/pull-from-upstream

# This builds the firmware for a machine of your choice
v1-scripts/build-for-machine MPCNC/Ramps_T8_16T_LCD_32step_DualEndstop -t upload --upload-port /dev/ttyACM0

NB! This is all preliminary and it messes around with Git and branches so don’t play with it in a repository where you have unpublished stuff that you care about.

EDIT: Updated script names to reflect the new state

Did you observe it yourself and more importantly, do you still have the board / LCD combination? Why I ask is that I spent half a day digging through ST7920 datasheet and talking to a few folks on discord and I think Marlin is over-driving these displays. I’ve ordered a cheap logic analyzer to confirm the timings on a set of boards I have but since I can’t reproduce any display issues on my RAMPS even without any delays it would be nice to rope in someone to test on an AVR board once I have figured out what changes are needed to correct it.

So should I update my current branches, and this will work with them?

Yes I should, I set one aside but I set a few aside so I will have to do a bit of hunting…I need to do this with the next update anyway. LCD and up the driver current on all boards.

Is this something I can do with an oscope? If you give me an idea of what I am looking for I can try to hook it up??? (clueless with that tool but it is fun to play with).

1 Like

I’m kind of hoping to just replace your current branches. This should take a vanilla version of Marlin, and for each machine configuration, create a new configuration.h/adv.h, save them to examples, and make .zips. You can then take the .zip and do any testing you want and release them.

Additionally, new functionality that happens for free is that it will do this every 6 hours if Marlin changes anything, so anyone can find a “bleeding edge” release and test it out (including yourself). One of the still todo items is defining a good way to keep those bleeding edge releases and the stable ones and making them obvious which is which.

I’m not sure what the next step is. I would love to try to get this running on the 2.0.1 release that just happened.

After you move to this system, you won’t edit the config files directly, you’d just edit the scripts to change any parameters.

4 Likes

Here’s a list of stuff that’s still open.

1. Decide if we pursue the inclusion of a single “sample” MPCNC config in Marlin upstream

If yes, let’s review Marlin/config/examples/CNC/MPCNC at v1-base-config · anttix/Marlin · GitHub and cut a PR to Marlin. The advantage of having a config there is that comments and formatting for more complex stuff e.g. JD and CNC settings will remain intact. We should also be able to also push a test that uses this config to reduce a chance of compile errors creeping into CNC code paths in the future. The disadvantage is that if we want to change the settings we should cut a PR to Marlin. We can also forget about this idea and go with 100% generated.

2. Implement version matrix

Right now the workflows only pull down bugfix-2.0.x. This needs to be extended to include at least 2.0.x and possibly dev-2.1.x as well.

3. Hone local development process

The new development process will look like this:

  1. Edit a script under v1-scripts/config
  2. Run generate-configs (or generate-build-branch if Marlin update is also involved)
  3. Run build-for-machine
  4. Rinse and repeat.

Right now there is a mandatory commit between steps 1 and 2 and a switch back to v1-machines branch between 3 and 4. This is sub-optimal for fast turnaround, there may be a better way. I’m looking into git submodules to see if there is a combination that would provide a better experience. Submodules have their own problems though.

4. Implement a release process

Github has releases and it also has a way of marking a “pre-release” so this is the functionality we should use to distinguish between releases. My current idea is that to do a release, one would push a copy to a tag: e.g. v415-rc1 and then the build process will automatically upload zips to a pre-release draft which can be reviewed, then published. Once RC-s have gone through testing, the final tree will be pushed to v415 tag. We also need a script similar to Marlin/buildroot/bin/generate_version at 2.0.x · MarlinFirmware/Marlin · GitHub that will automatically update the version number during builds.

5. Review new configs

Preliminary scripts to generate configs are at Marlin/v1-scripts/configs at v1-machines-demo · anttix/Marlin · GitHub

6. Sanity test

So far all of this stuff just compiles. Some brave soul should pull down one of the firmwares from Release unstable · anttix/Marlin · GitHub and see if it actually works. Bonus points for pointing octoprint firmware updater plugin directly to a .bin or a .hex file :wink:

I would think #1, #5, #6 and #3 are good candidates to knock out of the park next.

This is a bit off-topic so it may need a new topic but, here we go: There are two potential issues that I’ve discovered.

  1. Clock signal may be too fast. The datasheet specifies 200ns/300ns minimum times for both low and high states for SCLK pin. The difference is down to the supply voltage. Reprap wiki has schematics that specify 3V supply but I’m not sure how many existing displays follow that or if they skimp on the regulator and run at 5V. In any case 300ns is the safe value. My calculations off of Marlin code indicated that at least low state may be significantly shorter than this minimum especially on faster (20Mhz) boards.

  2. Commands may not have enough time to complete. Currently there is a default LCD delay of 10us but this is way lower than the datasheet “guaranteed” 72us. It seems that in practice the chips are a lot faster than the datasheet specifies though. The actual timings are unclear because there is a fair amount of code that runs between commands and cycle counting gets really old real quick. With an LA it should be easy to observe a quiet time between the “packets”.

Progress update: #3 is done. Marlin is modeled as a submodule now which makes it easier to work on config generators: No switching between branches needed.

I have looked through it a few times. I understand for the most part how you are doing things and what I would need to change. Cool! I still have not had a chance to do much with firmware. Got a bit of a bug from the holidays so I am pretty worn out the last few days.

1 Like

Yeah my whole family was sick for 2 days. I’m “eagerly” waiting for it to hit me …

1 Like

My wife and daughter got sick, and me and the boy haven’t. We did catch the nintendo christmas bug though :slight_smile:

1 Like

Like this one? https://i.redd.it/fziqjxoxzs641.jpg

3 Likes

As of yesterday, one can try this:

UPSTREAM_BRANCH=2.0.x ./v1-scripts/pull-from-upstream

Looks like it compiles, at least for RAMPS

2 Likes

@anttix I’m really impressed with all this work you have done! This is going to make getting my SKR board to Marlin 2.0.1 so much easier. In order to get my SKR 1.3 compiled with platformio 4 I had to update some of the marlin scripts to 2.0.x

From what I can see in your PR I’d just add another file and opt_{set|disable|enable} any of the custom configs? Is it really that simple or are there other things that need to be thought through.

3 Likes

Well I guess I didn’t need to do any work. My SKR 1.3 board is using DRV8825 and that’s all included in the default configuration. I was able to pull, build, and flash the firmware.

Though my Low Rider isn’t built yet, so I can’t say for sure if it’s working. Hopefully soon I can test my first plot/cut with this.

But seriously, this is so awesome! Bleeding edge of Marlin customized for CNC.

1 Like

I think I finally have one new firmware done. I kept track of all my changes. I i’ll start testing dual and other boards then I will try to move them to your setup with some PR’s.

I am nervous about conflicts though. I had one today from a minor change.

Thank you all. I was on a ski trip for the last week so I haven’t put much work into this.
Besides firing up 2.0.1 on my MPCNC and cutting out a few flat pieces as a test.

20200109_220431

Thank you for testing. I don’t think dual endstop firmware works on LR out of the box because it has dual Y,Z and single X while MPCNC has dual X,Y and single Z motor. I don’t have a LR myself yet but I think a simple 3 motor firmware works if motors are wired in series. I didn’t make such version for SKR but it’s relatively easy to add: There is a no-dual-endstops file that can be pulled in to make a 3 motor config.

Almost :slight_smile: Look at this commit. There are three files that need to be added/modified.

I’m almost done building my LR. Just need to build a table for it and then I can start testing this new firmware. I’ll let you know how it goes once I have a few cut under my belt.

Long time, no update. I’ve been busy bulding my lowrider with the little free time I had in January.

The movement in the meantime has been:

  1. @vicious1 contributed his latest tweaks to kinematic settings (junction deviation, accelerations etc). This is now merged
  2. Killed the “sample” MPCNC config idea: It’s 100% generated now.
  3. Marlin upstream broke the builds by moving configurations out of the main repository and breaking the scripts that were used to “apply” these configs - fixed now.
  4. Github made some changes that broke automatic pull from upstream - fixed now.

Overall I think out method of generating configs “aged” pretty well, as issues #3 and #4 took me less than 15min to correct and Marlin still compiles even after fairly large changes upstream.

I need to find some time to get the version script done and get the release process to a better place. There have been some github UI changes that make build “artifacts” easier to find, so maybe nightly “releases” are not required any more and nightlies can be modeled as build artifacts instead.

2 Likes

@dolfelt how is your build coming along? I just created a single-driver SKR1.3 config and a dual Y/Z one specifically for lowrider. Feel free to check these out.