From purpleidea at gmail.com Sat Dec 1 00:44:28 2018 From: purpleidea at gmail.com (James) Date: Fri, 30 Nov 2018 19:44:28 -0500 Subject: [mgmt] release: 0.0.16 (engine re-write and new resources) Message-ID: I've just released version 0.0.16 of mgmt! > 220 files changed, 14243 insertions(+), 9491 deletions(-) Woo... There's some great new stuff that landed since 0.0.15, including: * A giant engine re-write! (Makes resource writing more elegant too.) * New resources!! * New language features!!! and so much more... This is a monster release. Please try out the new features today :) NEWS * New resources include: net, mount and docker:container. Jonathan was responsible for all of these. Please take them for a spin! He's looking for a job too, and would probably be happy to get paid to work on mgmt. * We're > 1.5k stars on GitHub now. It's a silly metric, but ?\_(?)_/? * A giant engine refactoring/re-write was done. This cleaned up the code significantly, and made it more elegant to write resources. Unfortunately there is one small bug that I missed and that I haven't fixed yet. It rarely happens except during some of our tests during shutdown, which causes intermittent failures. It shouldn't block you playing with mgmt. * The language "class" and "include" statements have been added. These are important pieces for writing reusable modules which are coming soon. Try them out! (This comes with a bunch of tests too.) * We have an integration testing framework. It's pretty cool, it spins up a full mgmt cluster and runs stuff. Try it out or add some tests. * I had fun fixing a big bug: 06ee05026b0c743d19c7d62675f8ddeabdc8dd4f * I removed the remote execution functionality from core. I realized it could be re-written as a resource, and it was also in the way from some other cleanups that were more important. Half the new code is done, ping me if this is a priority for you or you want to help. * I also removed the HCL front-end, because mcl is usable enough to be more fun to play with, and I wanted to refactor some code. If someone really wants it back, let me know. * We have some release building scripts in git master, so you can now download pre-built (with fpm) RPM, DEB, or PACMAN packages! They're signed too. https://github.com/purpleidea/mgmt/releases/tag/0.0.16 * We're looking for help writing Google, DigitalOcean, Rackspace, etc resources if anyone is interested, reach out to us. Particularly if there is support from those organizations as well. * Many other bug fixes, changes, etc... * See the git log for more NEWS, and for anything notable I left out! MISC I took a bit of a break recently to catch up on some life stuff, but I think I'm back on track. While git master hasn't been especially busy, there's an active feature branch at feat/import which contains some fun stuff, with a very WIP giant patch still sitting on my machine. I hope to finish it up soon and then do another release. That branch contains one of the last big features before I'll really be ready to run mgmt on my personal servers! We're still looking for new contributors, and there are easy, medium and hard issues available! You're also welcome to suggest your own! Please join us in #mgmtconfig on Freenode IRC, or ping this list if you'd like help getting started! For details please see: https://github.com/purpleidea/mgmt/blob/master/docs/faq.md#how-do-i-con tribute-to-the-project-if-i-dont-know-golang Many new tagged #mgmtlove issues were tagged: https://github.com/purpleidea/mgmt/issues?q=is%3Aissue+is%3Aopen+label% 3Amgmtlove MENTORING We offer mentoring for new golang/mgmt hackers who want to get involved. This is free and friendly. You get to improve your skills, and we get some patches in return. Ping me off-list for details. THANKS Thanks (alphabetically) to everyone who contributed to the latest release: Alan Jenkins, James Shubin, jesus m. rodriguez, Jonathan Gold, jonathangold, Lauri Ojansivu, phaer We had 7 unique committers since 0.0.15, and have had 52 overall. run 'git log 0.0.15..0.0.16' to see what has changed since 0.0.15 Happy hacking, James @purpleidea -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: This is a digitally signed message part URL: From purpleidea at gmail.com Sat Dec 1 00:47:51 2018 From: purpleidea at gmail.com (James) Date: Fri, 30 Nov 2018 19:47:51 -0500 Subject: [mgmt] release: 0.0.16 (how I do releases) In-Reply-To: References: Message-ID: <721daa94246d25bbcffbe1435b639d37fe373393.camel@gmail.com> On Fri, 2018-11-30 at 19:44 -0500, James wrote: > * We have some release building scripts in git master, so you can now > download pre-built (with fpm) RPM, DEB, or PACMAN packages! They're > signed too. https://github.com/purpleidea/mgmt/releases/tag/0.0.16 If you'd like to see how I made a release, I've attached the terminal output from the process to this email if you're curious! Hope this is interesting! Cheers, James -------------- next part -------------- james at computer:~/code/mgmt$ make tag ./misc/tag.sh WARN: About to tag "0.0.15-61-g0ec00fe" as "0.0.16" and push. Press ^C within 3s to abort. You need a passphrase to unlock the secret key for user: "James Shubin (Third PGP key.) " 4096-bit RSA key, ID 24090D66, created 2012-05-09 INFO: Version 0.0.16 is now tagged. INFO: Pushing 0.0.16 to origin. Counting objects: 8, done. Delta compression using up to 4 threads. Compressing objects: 100% (8/8), done. Writing objects: 100% (8/8), 2.51 KiB | 1.25 MiB/s, done. Total 8 (delta 6), reused 0 (delta 0) remote: Resolving deltas: 100% (6/6), completed with 6 local objects. To github.com:purpleidea/mgmt.git * [new tag] 0.0.16 -> 0.0.16 .gitignore | 2 +- .gitmodules | 9 + .travis.yml | 3 + Makefile | 126 ++++++-- README.md | 7 +- Vagrantfile | 4 +- converger/converger.go | 102 ++++-- docs/documentation.md | 14 +- docs/language-guide.md | 137 +++++++- docs/on-the-web.md | 3 +- docs/quick-start-guide.md | 7 +- docs/resource-guide.md | 543 +++++++++++++++++++++----------- docs/resources.md | 48 ++- resources/uid.go => engine/autoedge.go | 51 ++- engine/autoedge_test.go | 38 +++ engine/autogroup.go | 84 +++++ engine/cmp.go | 126 ++++++++ {resources => engine}/edge.go | 18 +- engine/error.go | 32 ++ engine/event/event.go | 33 ++ resources/interfaces.go => engine/fs.go | 30 +- engine/graph/actions.go | 474 ++++++++++++++++++++++++++++ engine/graph/autoedge.go | 30 ++ {resources => engine/graph/autoedge}/autoedge.go | 167 +++++----- engine/graph/autogroup.go | 141 +++++++++ engine/graph/autogroup/autogroup.go | 71 +++++ {resources => engine/graph/autogroup}/autogroup_test.go | 338 +++++++++++++++----- engine/graph/autogroup/base.go | 127 ++++++++ engine/graph/autogroup/nonreachability.go | 73 +++++ engine/graph/autogroup/util.go | 127 ++++++++ engine/graph/engine.go | 336 ++++++++++++++++++++ engine/graph/refresh.go | 59 ++++ {resources => engine/graph}/semaphore.go | 10 +- resources/virt_disabled.go => engine/graph/semaphore_test.go | 22 +- engine/graph/sendrecv.go | 118 +++++++ engine/graph/state.go | 436 ++++++++++++++++++++++++++ engine/metaparams.go | 169 ++++++++++ engine/metaparams_test.go | 42 +++ engine/refresh.go | 32 ++ engine/resources.go | 271 ++++++++++++++++ {resources => engine/resources}/augeas.go | 151 +++++---- {resources => engine/resources}/aws_ec2.go | 536 +++++++++++++++---------------- engine/resources/docker_container.go | 441 ++++++++++++++++++++++++++ engine/resources/docker_container_test.go | 201 ++++++++++++ {resources => engine/resources}/exec.go | 358 +++++++++++---------- {resources => engine/resources}/exec_test.go | 42 +-- {resources => engine/resources}/file.go | 397 ++++++++++++----------- engine/resources/file_test.go | 148 +++++++++ {resources => engine/resources}/group.go | 161 +++++----- {resources => engine/resources}/hostname.go | 154 ++++----- {resources => engine/resources}/kv.go | 130 ++++---- engine/resources/mount.go | 732 ++++++++++++++++++++++++++++++++++++++++++ engine/resources/mount_test.go | 343 ++++++++++++++++++++ {resources => engine/resources}/msg.go | 148 +++++---- {resources => engine/resources}/msg_test.go | 14 +- engine/resources/net.go | 888 +++++++++++++++++++++++++++++++++++++++++++++++++++ {resources => engine/resources}/noop.go | 106 ++++--- engine/resources/noop_test.go | 104 ++++++ {resources => engine/resources}/nspawn.go | 254 +++++++-------- {resources => engine/resources}/packagekit/packagekit.go | 274 +++++++++------- {resources => engine/resources}/password.go | 163 ++++++---- {resources => engine/resources}/pkg.go | 235 +++++++------- {resources => engine/resources}/pkg_test.go | 2 + {resources => engine/resources}/print.go | 128 ++++---- {resources => engine/resources}/svc.go | 223 +++++++------ {resources => engine/resources}/test.go | 217 ++++++++----- engine/resources/test_test.go | 177 +++++++++++ {resources => engine/resources}/timer.go | 103 +++--- {resources => engine/resources}/user.go | 244 +++++++------- {resources => engine/resources}/virt.go | 225 ++++++------- engine/sendrecv.go | 67 ++++ engine/traits/autoedge.go | 42 +++ engine/traits/autogroup.go | 89 ++++++ engine/traits/base.go | 36 +++ engine/traits/kind.go | 39 +++ engine/traits/meta.go | 40 +++ engine/traits/named.go | 40 +++ engine/traits/refresh.go | 40 +++ engine/traits/sendrecv.go | 75 +++++ engine/util.go | 41 +++ {resources => engine/util}/util.go | 47 ++- engine/util/util_test.go | 107 +++++++ engine/world.go | 48 +++ etcd/etcd.go | 31 +- etcd/etcd_test.go | 2 + {event => etcd/event}/event.go | 49 --- etcd/fs/fs_test.go | 2 + etcd/resources.go | 29 +- etcd/world.go | 14 +- examples/lang/docker_container0.mcl | 6 + examples/lang/mount0.mcl | 5 + examples/lang/mount1.mcl | 5 + examples/lang/mount2.mcl | 13 + examples/lang/net1.mcl | 5 + examples/lang/net2.mcl | 3 + examples/lang/net3.mcl | 3 + examples/lib/exec-send-recv.go | 59 ++-- examples/lib/libmgmt-subgraph0.go | 31 +- examples/lib/libmgmt-subgraph1.go | 40 +-- examples/lib/libmgmt1.go | 246 --------------- examples/lib/libmgmt2.go | 8 +- examples/lib/libmgmt3.go | 44 +-- gapi/deploy.go | 2 +- hcl/gapi.go => gapi/empty/empty.go | 137 +++----- gapi/gapi.go | 6 +- gapi/helpers.go | 8 +- hcl/hil/interpolate.go | 89 ------ hcl/parse.go | 381 ---------------------- integration/basic_test.go | 267 ++++++++++++++++ integration/cluster.go | 224 +++++++++++++ integration/instance.go | 392 +++++++++++++++++++++++ integration/patterns.go | 135 ++++++++ integration/util.go | 65 ++++ integration/util_test.go | 62 ++++ lang/funcs/contains_polyfunc.go | 4 +- lang/funcs/core/exchange_polyfunc.go | 2 +- lang/funcs/core/kvlookup_func.go | 2 +- lang/funcs/core/schedule_polyfunc.go | 4 +- lang/funcs/engine.go | 66 +++- lang/funcs/facts/facts.go | 4 +- lang/funcs/facts/func_test.go | 2 + lang/funcs/funcs.go | 10 +- lang/funcs/funcs_test.go | 2 + lang/funcs/maplookup_polyfunc.go | 4 +- lang/funcs/operator_polyfunc.go | 4 +- lang/funcs/simple/env_func.go | 4 +- lang/funcs/simplepoly/len_polyfunc.go | 2 +- lang/funcs/simplepoly/simplepoly.go | 4 +- lang/funcs/structlookup_polyfunc.go | 4 +- lang/gapi.go | 56 +++- lang/interfaces/ast.go | 21 +- lang/interfaces/func.go | 4 +- lang/interpolate_test.go | 2 + lang/interpret.go | 54 ++-- lang/interpret_test.go | 15 +- lang/lang.go | 52 +-- lang/lang_test.go | 525 +++++++++++++++++++++++++++++-- lang/lexer.nex | 15 + lang/lexparse.go | 1 + lang/lexparse_test.go | 220 ++++++++++++- lang/parser.y | 92 +++++- lang/structs.go | 437 ++++++++++++++++++++++---- lang/types/type.go | 6 +- lang/types/type_test.go | 32 +- lang/types/value.go | 2 +- lang/types/value_test.go | 18 +- lang/unification_test.go | 4 +- lib/cli.go | 116 +++---- lib/converged.go | 38 +++ lib/deploy.go | 3 - lib/hello.go | 2 +- lib/main.go | 615 ++++++++++++++++++------------------ lib/main_test.go | 38 +++ main.go | 12 +- misc/fpm-pack.sh | 76 +++++ misc/gen-deb-changelog-from-git.sh | 76 ----- misc/make-deb-changelog.sh | 48 +++ misc/make-deps.sh | 11 +- misc/make-rpm-changelog.sh | 48 +++ tag.sh => misc/tag.sh | 0 pgraph/graphsync.go | 2 +- pgraph/graphsync_test.go | 2 + pgraph/pgraph.go | 46 +-- pgraph/pgraph_test.go | 54 +++- pgraph/subgraph.go | 3 + pgraph/subgraph_test.go | 2 + pgraph/util_test.go | 2 + prometheus/prometheus_test.go | 2 + puppet/gapi.go | 12 +- puppet/puppet.go | 56 ++-- recwatch/configwatch.go | 148 --------- recwatch/recwatch.go | 20 +- remote/remote.go | 1116 ----------------------------------------------------------------- resources/actions.go | 649 -------------------------------------- resources/autogroup.go | 354 --------------------- resources/file_test.go | 79 ----- resources/graph.go | 234 -------------- resources/metaparams.go | 65 ---- resources/mgraph.go | 215 ------------- resources/refresh.go | 104 ------ resources/resources.go | 675 --------------------------------------- resources/resources_test.go | 145 --------- resources/semaphore_test.go | 107 ------- resources/sendrecv.go | 261 --------------- resources/util_test.go | 380 ---------------------- test.sh | 22 +- test/shell/env0.sh | 8 + test/shell/exec-fail.sh | 2 +- test/shell/exec-usergroup.sh | 2 +- test/shell/file-move.sh | 15 +- test/shell/graph-exit1.sh | 4 +- test/shell/libmgmt-change1.go | 4 +- test/shell/libmgmt-change2.go | 8 +- test/shell/load0.sh | 10 + test/shell/prometheus-3.sh | 2 + test/shell/prometheus-4.sh | 2 + test/test-commit-message.sh | 12 +- test/test-crossbuild.sh | 9 + test/test-examples.sh | 2 + test/test-gotest.sh | 35 ++- test/test-govet.sh | 28 ++ test/test-integration.sh | 13 + test/test-shell.sh | 3 + test/test-vet.sh | 54 ++++ util/code.go | 53 ++++ util/code_test.go | 47 +++ util/home.go | 55 ++++ resources/virt_test.go => util/home_test.go | 10 +- resources/augeas_disabled.go => util/root_test.go | 22 +- util/sync.go | 87 +++++ util/util.go | 25 ++ util/util_test.go | 48 ++- vagrant/motd | 2 +- vendor/github.com/docker/distribution | 1 + vendor/github.com/docker/docker | 1 + vendor/github.com/docker/go-connections | 1 + yamlgraph/gapi.go | 31 +- yamlgraph/gconfig.go | 277 +++++++++------- yamlgraph2/gapi.go | 205 ------------ yamlgraph2/gconfig.go | 311 ------------------ 220 files changed, 14243 insertions(+), 9491 deletions(-) Thanks (alphabetically) to everyone who contributed to the latest release: Alan Jenkins, James Shubin, jesus m. rodriguez, Jonathan Gold, jonathangold, Lauri Ojansivu, phaer We had 7 unique committers since 0.0.15, and have had 52 overall. run 'git log 0.0.15..0.0.16' to see what has changed since 0.0.15 james at computer:~/code/mgmt$ make release Generating: bindata... Generating: lang... make --quiet -C lang mkdir -p releases/0.0.16/{deb,rpm,pacman}/ && touch releases/0.0.16/.mkdir Generating rpm changelog... ./misc/make-rpm-changelog.sh 0.0.16 Building rpm package... ./misc/fpm-pack.sh rpm 0.0.16 libvirt-devel augeas-devel Created package {:path=>"releases/0.0.16/rpm/mgmt-0.0.16-1.x86_64.rpm"} Generating deb changelog... ./misc/make-deb-changelog.sh 0.0.16 Building deb package... ./misc/fpm-pack.sh deb 0.0.16 libvirt-dev libaugeas-dev Created package {:path=>"releases/0.0.16/deb/mgmt_0.0.16_amd64.deb"} Building pacman package... ./misc/fpm-pack.sh pacman 0.0.16 libvirt augeas Created package {:path=>"releases/0.0.16/pacman/mgmt-0.0.16-1-x86_64.pkg.tar.xz"} Generating sha256 sum... sha256sum releases/0.0.16/rpm/mgmt-0.0.16-1.x86_64.rpm releases/0.0.16/deb/mgmt_0.0.16_amd64.deb releases/0.0.16/pacman/mgmt-0.0.16-1-x86_64.pkg.tar.xz | awk -F '/| ' '{print $1" "$6}' > releases/0.0.16/SHA256SUMS Signing sha256 sum... gpg2 --yes --clearsign releases/0.0.16/SHA256SUMS gpg: using "A0E8F3C024090D66" as default secret key for signing Creating github release... hub release create \ -F <( echo -e "0.0.16\n";echo "Verify the signatures of all packages before you use them. The signing key can be downloaded from https://purpleidea.com/contact/#pgp-key to verify the release." ) \ -a releases/0.0.16/rpm/mgmt-0.0.16-1.x86_64.rpm \ -a releases/0.0.16/deb/mgmt_0.0.16_amd64.deb \ -a releases/0.0.16/pacman/mgmt-0.0.16-1-x86_64.pkg.tar.xz \ -a releases/0.0.16/SHA256SUMS.asc \ 0.0.16 \ > releases/0.0.16/mgmt-release.url \ && cat releases/0.0.16/mgmt-release.url \ || rm -f releases/0.0.16/mgmt-release.url Attaching release asset `releases/0.0.16/rpm/mgmt-0.0.16-1.x86_64.rpm'... Attaching release asset `releases/0.0.16/deb/mgmt_0.0.16_amd64.deb'... Attaching release asset `releases/0.0.16/pacman/mgmt-0.0.16-1-x86_64.pkg.tar.xz'... Attaching release asset `releases/0.0.16/SHA256SUMS.asc'... https://github.com/purpleidea/mgmt/releases/tag/0.0.16 james at computer:~/code/mgmt$ -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: This is a digitally signed message part URL: