New juju-mongodb package

Discussion:

James Page

2013-11-28 13:46:55 UTC

Hi Folks

I've started working on the new, stripped down, juju specific MongoDB
package that we have been discussing over the last few weeks.

I'm proposing a package structure like this:

./usr/lib/juju/bin/mongos
./usr/lib/juju/bin/mongod

No users will be created; its just the binaries; upstart and general
system configuration such as creating users will be the responsibility
of juju.

The mongod and mongos binaries will be provided in a juju namespaced
location to avoid conflicting with the standard mongodb package; v8
will be linked statically using the embedded copy of v8 in the mongodb
source code - this avoids exposing v8 generally in main and allows the
security team to manage mongodb/v8 in the context of its use with
juju, rather than in more broad general use.

The plan is that we will apply for a minor release exception for this
package, and that if need be we can update to a new major release (2.6
for example) at some point in the future without impacting the rest of
the distro by bumping the standard mongodb package.

The total compressed package size is about 7MB - expanding to about
23MB on disk.

I still need todo some work on getting the embedded v8 copy to build
for armhf (MongoDB upstream strip this out) - arm64 has been discussed
but that's going to need some work upstream to enable v8 for this
architecture.

Other bugs pertinent MongoDB/juju usage would include:

https://bugs.launchpad.net/juju-core/+bug/1208430

I'm pretty sure that running mongodb not as root will be part of the
security team signoff on the MIR review.

Cheers

James

- --
James Page
Technical Lead
Ubuntu Server Team
james.page at canonical.com

Gustavo Niemeyer

2013-11-28 14:18:46 UTC

Permalink

Thanks for pushing this, James.

It would be good to have the "mongo" binary available and working as
well, also under that juju-specific namespace. This is the console
client, and will be useful to connect to the local juju database when
debugging issues.

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256
Hi Folks
I've started working on the new, stripped down, juju specific MongoDB
package that we have been discussing over the last few weeks.
./usr/lib/juju/bin/mongos
./usr/lib/juju/bin/mongod
No users will be created; its just the binaries; upstart and general
system configuration such as creating users will be the responsibility
of juju.
The mongod and mongos binaries will be provided in a juju namespaced
location to avoid conflicting with the standard mongodb package; v8
will be linked statically using the embedded copy of v8 in the mongodb
source code - this avoids exposing v8 generally in main and allows the
security team to manage mongodb/v8 in the context of its use with
juju, rather than in more broad general use.
The plan is that we will apply for a minor release exception for this
package, and that if need be we can update to a new major release (2.6
for example) at some point in the future without impacting the rest of
the distro by bumping the standard mongodb package.
The total compressed package size is about 7MB - expanding to about
23MB on disk.
I still need todo some work on getting the embedded v8 copy to build
for armhf (MongoDB upstream strip this out) - arm64 has been discussed
but that's going to need some work upstream to enable v8 for this
architecture.
https://bugs.launchpad.net/juju-core/+bug/1208430
I'm pretty sure that running mongodb not as root will be part of the
security team signoff on the MIR review.
Cheers
James
- --
James Page
Technical Lead
Ubuntu Server Team
james.page at canonical.com
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.15 (GNU/Linux)
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/
iQIcBAEBCAAGBQJSl0lPAAoJEL/srsug59jDwrkQAKQuDJsm5I8YPeIwzTv+4/Gd
mcIQHcPer3EHaMB9/4zjWg8EoNuU0dj52PCjEEbI6zBZek1VqqxYLbsLxdzoo6x1
SAVulHOCG/oVcjL/XVQ7EYTodZrXOEAYKlAOOxI2pj9ea6XoQVI3i4SsZdMCNUyr
+CBUKrxH8YPcM3mXIyJBw0qbiHFLJQsywC1gZnsLTomox2Ob+eIk+n/CH2d0tMv3
DyD5c5GAypnHzmrsiteJuPu01OqMaYsiltRWaFFEzuV7C8eIVW4uRFowC+xX8a0i
UZ6FTUri4l9F4OahoJdVyjTofgQuis6pa/uKQWp7AUA+40JB/uMKApNV1xJ1772c
8YXVYNVYEc9r+x/LzuqFyHt1BEdqYgDt4ZG7mYR0AwW4p4eK7wAFRElH8JPVdEQ9
9iDFV/FFFCcWdVKRKSUuDCdv3bCNnEOLZU2qD0db9IPUNfGGeedkrSJAXGxBiCkg
9OBxp4xylrbU4gw9tJDmicctIXG6N+n/XMlsDj5FkWqmGAq3HqdXw6VCPJ48X7+m
ZHnXoQniR0eoh021xxFmb+f4eTG6U9YY8oyefMERLliVD/a26qQ6VQDa0M1mJig1
OQzbJoaXbuJulew7B+sFI0ltMtx6CVhtyXobKATEKMrzs5GcqT15B7+K5W0Uisca
Wp2ySVEZGOc3Sv3fobBC
=XBGC
-----END PGP SIGNATURE-----
--
Juju-dev mailing list
Juju-dev at lists.ubuntu.com
Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/juju-dev

--
gustavo @ http://niemeyer.net

James Page

2013-11-28 14:38:32 UTC

Permalink

Post by Gustavo Niemeyer
It would be good to have the "mongo" binary available and working
as well, also under that juju-specific namespace. This is the
console client, and will be useful to connect to the local juju
database when debugging issues.

Makes sense - added.

- --
James Page
Technical Lead
Ubuntu Server Team
james.page at canonical.com

William Reade

2013-11-28 17:53:06 UTC

Permalink

The mongodump and mongorestore binaries would also be extremely useful.

Cheers
William

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

Makes sense - added.
- --
James Page
Technical Lead
Ubuntu Server Team
james.page at canonical.com
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.15 (GNU/Linux)
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/
iQIcBAEBCAAGBQJSl1VnAAoJEL/srsug59jDDeoQAL9mi/qX8QlC9sx0i3MTjT4W
N8VlvReHg3bgYlig5qZVu6ShMiZ3vd6mY9qoFB+FdVEmMEazHr5+am/JgmLAF+6G
nOEAwsiZSBPO/pMOypXPdp0uvSlL8dmCN3aXzQ8X7R0hWZhm1dHhk6DHaY+VImIr
pjs31OaRpCgxqgAjwYcw76apVb0gnKNrDwYA45lPDPMlufFUe7l1xlGJJJFPj8Ap
9QC8Jzy/LeYagb3D4SeMEAEnpSg2nuoQr1KXVbb87RPPUy6XAFWfyTNyaLmMgypG
Ci254fcC6U22gFLyVj4A5oXSFR2a3tLEEkubN4qJHMVXRFJm2OxZZrTSnNpnUiOW
9vSQmteLUxMMKa6RugZRNJFPWZBu4Sg6O/uV5/y9Lv675sVfJy+j12N1RxAHXmae
MZgsKj7F7LIOghB18bJvJh6r2FUCZc5l7pywLUysCtEWw1lCBTkCHe1QqCpRR/Fu
r2w85OAM4IOdrtyMa1mpt0G78jJgq7wvyFifI+MXo/WvTmE8G7EnKDvreN76z0FI
jagTKvSgqqRQ91tNjiUkFuKKbji/KBjOjmXf88ho3pmMfYRSm/LZhytBNgfxMiKs
XVWc9t+MFOcs0GINdh9pYJQf+9SE7XpTkLkLcQ9QpgDVFmZpu8trkec40WOxJ4nq
AcybyPpBCHO5wjyQ+pvi
=++qm
-----END PGP SIGNATURE-----
--
Juju-dev mailing list
Juju-dev at lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/juju-dev

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.ubuntu.com/archives/juju-dev/attachments/20131128/5a7460df/attachment.html>

Ian Booth

2013-11-28 20:10:05 UTC

Permalink

Also mongoexport (used in the new juju backup plugin)

Post by William Reade
The mongodump and mongorestore binaries would also be extremely useful.
Cheers
William
********* *BEGIN ENCRYPTED or SIGNED PART* *********

Makes sense - added.
--
James Page
Technical Lead
Ubuntu Server Team
james.page at canonical.com
********** *END ENCRYPTED or SIGNED PART* **********

Post by Gustavo Niemeyer
--
Juju-dev mailing list
Juju-dev at lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/juju-dev

--
Juju-dev mailing list
Juju-dev at lists.ubuntu.com
Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/juju-dev

James Page

2013-12-02 09:01:49 UTC

Permalink

Morning folks

so far I have requests for:

mongo (default shell)

mongoexport (used in new juju backup plugin)

mongodump/mongorestore (useful)

I'm keen we don't bloat out the 'server' package to much - the
binaries are quite large due to the requirement to statically link v8.

Which of these is core to the function of juju, and which are useful
for debugging? Please also that the mongodb-clients package will
still be available in universe so you will always be able to get any
of the client tools that way as well (I'll probably keep those
packages maintained to the latest point release for the released series).

Post by Ian Booth
Also mongoexport (used in the new juju backup plugin)

Post by William Reade
The mongodump and mongorestore binaries would also be extremely useful.
Cheers William
On Thu, Nov 28, 2013 at 3:38 PM, James Page
********* *BEGIN ENCRYPTED or SIGNED PART* *********

Post by Gustavo Niemeyer
It would be good to have the "mongo" binary available and
working as well, also under that juju-specific namespace. This
is the console client, and will be useful to connect to the
local juju database when debugging issues.

Makes sense - added.
-- James Page Technical Lead Ubuntu Server Team
james.page at canonical.com
********** *END ENCRYPTED or SIGNED PART* **********

Post by Gustavo Niemeyer
-- Juju-dev mailing list Juju-dev at lists.ubuntu.com Modify
https://lists.ubuntu.com/mailman/listinfo/juju-dev

-- Juju-dev mailing list Juju-dev at lists.ubuntu.com Modify
https://lists.ubuntu.com/mailman/listinfo/juju-dev

- --
James Page
Technical Lead
Ubuntu Server Team
james.page at canonical.com

John Arbash Meinel

2013-12-02 09:10:03 UTC

Permalink

Post by James Page
Morning folks
mongo (default shell)
mongoexport (used in new juju backup plugin)
mongodump/mongorestore (useful)
I'm keen we don't bloat out the 'server' package to much - the
binaries are quite large due to the requirement to statically link v8.
Which of these is core to the function of juju, and which are
useful for debugging? Please also that the mongodb-clients package
will still be available in universe so you will always be able to
get any of the client tools that way as well (I'll probably keep
those packages maintained to the latest point release for the
released series).

Is mongodump/mongorestore going to be in the mongodb-client or
mongodb-server?

How is this going to relate if we have to release a custom version of
MongoDB (2.6 instead of 2.4).

The only thing we actively use on a regular basis (AFAIK) is mongos
and mongod. However, if we set up regular backup process, then we'll
need one of mongodump or mongoexport (I don't know what the difference
is between them).
And obviously dump isn't very useful if you can't restore, though at
the point you are restoring is a reasonable time to create new tools.

I don't fully understand the difference from mongoexport vs mongodump
(it looks like dump generates a binary snapshot compatible with
restore, while export/import generate text representations of the data.)

I'd *really* like us to stick with *one* of them as the recommended
method for backing up the content.

At which point, mongos, monogod and mongoexport/mongodump become
critical for the regular operation of the system (assuming we set up
backups as a regular process), and then mongo,
mongoimport/mongorestore become tools that you install when you want
to inspect/restore/probe/etc.

If you're happier splitting them, I'm personally fine with that. I
*am* a bit concerned about them being split and then we bump the
version of juju-mongodb and then we end up without the corresponding
tools available.

John
=:->

James Page

2013-12-02 10:31:53 UTC

Permalink

Post by John Arbash Meinel

Post by James Page
Morning folks
mongo (default shell)
mongoexport (used in new juju backup plugin)
mongodump/mongorestore (useful)
I'm keen we don't bloat out the 'server' package to much - the
binaries are quite large due to the requirement to statically
link v8.
Which of these is core to the function of juju, and which are
useful for debugging? Please also that the mongodb-clients
package will still be available in universe so you will always be
able to get any of the client tools that way as well (I'll
probably keep those packages maintained to the latest point
release for the released series).

Is mongodump/mongorestore going to be in the mongodb-client or
mongodb-server?

Currently I was just planning a juju-mongodb package; however we could
split things that need to be on the client end of juju into a
juju-mongodb-clients package.

Post by John Arbash Meinel
How is this going to relate if we have to release a custom version
of MongoDB (2.6 instead of 2.4).

Yeah - that is tricky; that's why I was asking what do we need, rather
than what is nice to have; I would suspect that mongo shell is
probably forward compatible for example but that would not be nice if
it wasn't.

Post by John Arbash Meinel
The only thing we actively use on a regular basis (AFAIK) is
mongos and mongod. However, if we set up regular backup process,
then we'll need one of mongodump or mongoexport (I don't know what
the difference is between them). And obviously dump isn't very
useful if you can't restore, though at the point you are restoring
is a reasonable time to create new tools.
I don't fully understand the difference from mongoexport vs
mongodump (it looks like dump generates a binary snapshot
compatible with restore, while export/import generate text
representations of the data.)
I'd *really* like us to stick with *one* of them as the
recommended method for backing up the content.

+1 - we can add them all but it bloats out a bit.

Post by John Arbash Meinel
At which point, mongos, monogod and mongoexport/mongodump become
critical for the regular operation of the system (assuming we set
up backups as a regular process), and then mongo,
mongoimport/mongorestore become tools that you install when you
want to inspect/restore/probe/etc.
If you're happier splitting them, I'm personally fine with that. I
*am* a bit concerned about them being split and then we bump the
version of juju-mongodb and then we end up without the
corresponding tools available.

Installed sizes (taken from saucy which has the same v8 embedding):

- -rwxr-xr-x 1 root root 7.2M Sep 5 15:30 /usr/bin/mongo
- -rwxr-xr-x 1 root root 13M Sep 5 15:30 /usr/bin/mongodump
- -rwxr-xr-x 1 root root 13M Sep 5 15:30 /usr/bin/mongoexport
- -rwxr-xr-x 1 root root 13M Sep 5 15:30 /usr/bin/mongoimport
- -rwxr-xr-x 1 root root 13M Sep 5 15:30 /usr/bin/mongorestore

Expect about ~50% compression;

- --
James Page
Technical Lead
Ubuntu Server Team
james.page at canonical.com

Ian Booth

2013-12-02 11:40:11 UTC

Permalink

Post by John Arbash Meinel
I don't fully understand the difference from mongoexport vs mongodump
(it looks like dump generates a binary snapshot compatible with
restore, while export/import generate text representations of the data.)
I'd *really* like us to stick with *one* of them as the recommended
method for backing up the content.

Both export and dump are used by the Juju backup tool.
dump is used to do a full bson export of the database and is required for a
database recovery.
export is used to write to json the environment settings document so that there
is a human readable copy of the full environment settings contained in the
backup tarball.

roger peppe

2013-12-02 12:39:29 UTC

Permalink

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

ISTM that the mongo command could be used to similar effect, as John
suggests, couldn't it? Then we would not need mongexport
at all.

John Arbash Meinel

2013-12-02 12:48:35 UTC

Permalink

Post by roger peppe

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1

Post by John Arbash Meinel
I don't fully understand the difference from mongoexport vs
mongodump (it looks like dump generates a binary snapshot
compatible with restore, while export/import generate text
representations of the data.)
I'd *really* like us to stick with *one* of them as the
recommended method for backing up the content.

Both export and dump are used by the Juju backup tool. dump is
used to do a full bson export of the database and is required for
a database recovery. export is used to write to json the
environment settings document so that there is a human readable
copy of the full environment settings contained in the backup
tarball.

ISTM that the mongo command could be used to similar effect, as
John suggests, couldn't it? Then we would not need mongexport at
all.

For what we are using this page:

http://stackoverflow.com/questions/11255630/how-to-export-all-collection-in-mongodb

has something like:
mongo --eval 'printjson(db.getCollectionNames())'

We can easily change the internal one to something like
db.find('session') or whatever it needs to be.

Now, we wouldn't need 'mongo' otherwise, but I think if we have to
pick a tool 'mongo' is more generally useful, and it doesn't bundle v8
so it is a smaller binary as well.

So +1 to switching to 'mongo' instead of 'mongoexport'.

John
=:->

roger peppe

2013-12-02 13:05:05 UTC

Permalink

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Post by roger peppe

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1

Post by John Arbash Meinel
I don't fully understand the difference from mongoexport vs
mongodump (it looks like dump generates a binary snapshot
compatible with restore, while export/import generate text
representations of the data.)
I'd *really* like us to stick with *one* of them as the
recommended method for backing up the content.

Both export and dump are used by the Juju backup tool. dump is
used to do a full bson export of the database and is required for
a database recovery. export is used to write to json the
environment settings document so that there is a human readable
copy of the full environment settings contained in the backup
tarball.

ISTM that the mongo command could be used to similar effect, as
John suggests, couldn't it? Then we would not need mongexport at
all.

http://stackoverflow.com/questions/11255630/how-to-export-all-collection-in-mongodb
mongo --eval 'printjson(db.getCollectionNames())'
We can easily change the internal one to something like
db.find('session') or whatever it needs to be.
Now, we wouldn't need 'mongo' otherwise, but I think if we have to
pick a tool 'mongo' is more generally useful, and it doesn't bundle v8
so it is a smaller binary as well.

We will probably use the mongo command for restoring,
and may use it for backing up if that's a convenient way
to use fsyncLock to avoid taking down the whole
system for backing up.

And it's potentially useful for escaping awkward situations
too, so I'm definitely in favour of its inclusion.

Which brings us to {mongod, mongo, mongodump, mongorestore}
as the set of executables that we'd like to have, I think.

cheers,
rog.

John Arbash Meinel

2013-12-02 13:52:40 UTC

Permalink

...

Post by John Arbash Meinel
Now, we wouldn't need 'mongo' otherwise, but I think if we have
to pick a tool 'mongo' is more generally useful, and it doesn't
bundle v8 so it is a smaller binary as well.

We will probably use the mongo command for restoring, and may use
it for backing up if that's a convenient way to use fsyncLock to
avoid taking down the whole system for backing up.
And it's potentially useful for escaping awkward situations too, so
I'm definitely in favour of its inclusion.
Which brings us to {mongod, mongo, mongodump, mongorestore} as the
set of executables that we'd like to have, I think.
cheers, rog.

mongos, but otherwise I agree. We'll need a patch to juju bootstrap.

John
=:->

Kapil Thangavelu

2013-12-02 16:51:47 UTC

Permalink

Isn't what mongo and mongo export are doing trivially avail to a client API
script for juju backup purposes.

On 2 December 2013 12:48, John Arbash Meinel <john at arbash-meinel.com<javascript:;>>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 2 December 2013 11:40, Ian Booth <ian.booth at canonical.com<javascript:;>

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1

Post by John Arbash Meinel
I don't fully understand the difference from mongoexport vs
mongodump (it looks like dump generates a binary snapshot
compatible with restore, while export/import generate text
representations of the data.)
I'd *really* like us to stick with *one* of them as the
recommended method for backing up the content.

Both export and dump are used by the Juju backup tool. dump is
used to do a full bson export of the database and is required for
a database recovery. export is used to write to json the
environment settings document so that there is a human readable
copy of the full environment settings contained in the backup
tarball.

ISTM that the mongo command could be used to similar effect, as
John suggests, couldn't it? Then we would not need mongexport at
all.

http://stackoverflow.com/questions/11255630/how-to-export-all-collection-in-mongodb

mongo --eval 'printjson(db.getCollectionNames())'
We can easily change the internal one to something like
db.find('session') or whatever it needs to be.
Now, we wouldn't need 'mongo' otherwise, but I think if we have to
pick a tool 'mongo' is more generally useful, and it doesn't bundle v8
so it is a smaller binary as well.

We will probably use the mongo command for restoring,
and may use it for backing up if that's a convenient way
to use fsyncLock to avoid taking down the whole
system for backing up.
And it's potentially useful for escaping awkward situations
too, so I'm definitely in favour of its inclusion.
Which brings us to {mongod, mongo, mongodump, mongorestore}
as the set of executables that we'd like to have, I think.
cheers,
rog.
--
Juju-dev mailing list
Juju-dev at lists.ubuntu.com <javascript:;>
https://lists.ubuntu.com/mailman/listinfo/juju-dev

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.ubuntu.com/archives/juju-dev/attachments/20131202/8c930402/attachment.html>

John Arbash Meinel

2013-12-02 09:15:29 UTC

Permalink

Looking at the juju-backup script, it is intended to use both dump and
export. The export is so that we end up with JSON format for our
"settings" table so we can extract content from there as part of
restore (what credentials were you using, etc).

Are they both really big? If so, we can probably use: mongo --eval
"STUFF" instead of "mongoexport", but that would again presume that we
have a 'mongo' binary available.

I *believe* the data in the mongo dump is just BSON encoded, so it
would be possible to post-filter it with a bson aware tool to get it
into human-consumable form (rather than using mongoexport at backup time).

I guess the question is, how expensive is it to include it? For Juju
core folks the cost of including it is all externality so *we* aren't
particularly motivated to keep our toolset minimal. But if there is a
use case we want to support, we're willing to help.

John
=:->

James Page

2013-12-02 10:36:07 UTC

Permalink

Post by John Arbash Meinel

Post by James Page
Which of these is core to the function of juju, and which are
useful for debugging? Please also that the mongodb-clients
package will still be available in universe so you will always
be able to get any of the client tools that way as well (I'll
probably keep those packages maintained to the latest point
release for the released series).

Looking at the juju-backup script, it is intended to use both dump
and export. The export is so that we end up with JSON format for
our "settings" table so we can extract content from there as part
of restore (what credentials were you using, etc).
Are they both really big? If so, we can probably use: mongo --eval
"STUFF" instead of "mongoexport", but that would again presume that
we have a 'mongo' binary available.
I *believe* the data in the mongo dump is just BSON encoded, so it
would be possible to post-filter it with a bson aware tool to get
it into human-consumable form (rather than using mongoexport at
backup time).
I guess the question is, how expensive is it to include it? For
Juju core folks the cost of including it is all externality so *we*
aren't particularly motivated to keep our toolset minimal. But if
there is a use case we want to support, we're willing to help.

Installed sizes:

- -rwxr-xr-x 1 root root 7.2M Sep 5 15:30 /usr/bin/mongo
- -rwxr-xr-x 1 root root 13M Sep 5 15:30 /usr/bin/mongodump
- -rwxr-xr-x 1 root root 13M Sep 5 15:30 /usr/bin/mongoexport
- -rwxr-xr-x 1 root root 13M Sep 5 15:30 /usr/bin/mongoimport
- -rwxr-xr-x 1 root root 13M Sep 5 15:30 /usr/bin/mongorestore

Expect ~50% ish compression for the package itself.

Lets think about download sizes/times for end-users in this context
please :-). It costs time for every server that's provisioned.

- --
James Page
Technical Lead
Ubuntu Server Team
james.page at canonical.com

James Page

2013-12-05 16:06:00 UTC

Permalink

Post by John Arbash Meinel

Post by James Page
Which of these is core to the function of juju, and which are
useful for debugging? Please also that the mongodb-clients
package will still be available in universe so you will
always be able to get any of the client tools that way as
well (I'll probably keep those packages maintained to the
latest point release for the released series).

Looking at the juju-backup script, it is intended to use both
dump and export. The export is so that we end up with JSON format
for our "settings" table so we can extract content from there as
part of restore (what credentials were you using, etc).
Are they both really big? If so, we can probably use: mongo
--eval "STUFF" instead of "mongoexport", but that would again
presume that we have a 'mongo' binary available.
I *believe* the data in the mongo dump is just BSON encoded, so
it would be possible to post-filter it with a bson aware tool to
get it into human-consumable form (rather than using mongoexport
at backup time).
I guess the question is, how expensive is it to include it? For
Juju core folks the cost of including it is all externality so
*we* aren't particularly motivated to keep our toolset minimal.
But if there is a use case we want to support, we're willing to
help.

-rwxr-xr-x 1 root root 7.2M Sep 5 15:30 /usr/bin/mongo -rwxr-xr-x
1 root root 13M Sep 5 15:30 /usr/bin/mongodump -rwxr-xr-x 1 root
root 13M Sep 5 15:30 /usr/bin/mongoexport -rwxr-xr-x 1 root root
13M Sep 5 15:30 /usr/bin/mongoimport -rwxr-xr-x 1 root root 13M
Sep 5 15:30 /usr/bin/mongorestore
Expect ~50% ish compression for the package itself.
Lets think about download sizes/times for end-users in this
context please :-). It costs time for every server that's
provisioned.

Compressed package size comes to 24MB.

- --
James Page
Technical Lead
Ubuntu Server Team
james.page at canonical.com

John Arbash Meinel

2013-12-05 19:13:58 UTC

Permalink

...

Post by James Page

Post by James Page
-rwxr-xr-x 1 root root 7.2M Sep 5 15:30 /usr/bin/mongo
-rwxr-xr-x 1 root root 13M Sep 5 15:30 /usr/bin/mongodump
-rwxr-xr-x 1 root root 13M Sep 5 15:30 /usr/bin/mongoexport
-rwxr-xr-x 1 root root 13M Sep 5 15:30 /usr/bin/mongoimport
-rwxr-xr-x 1 root root 13M Sep 5 15:30 /usr/bin/mongorestore
Expect ~50% ish compression for the package itself.
Lets think about download sizes/times for end-users in this
context please :-). It costs time for every server that's
provisioned.

Compressed package size comes to 24MB.

So the discussion ended up that we could probably get away with a
minimum of:
mongo (requires running server, but doesn't bundle v8 itself)
mongod
mongodump
mongos

We can use 'mongo' instead of 'mongoexport' for the specific use case
(just have to run it before stopping the db).

We'll definitely want easy access to mongorestore, though I don't
think we need mongoimport. I get the feeling if we do have a
'juju-mongodb' which has our minimum set, we probably would still want
the possibility of installing 'juju-mongodb-tools' which has all of
mongoexport/mongoimport/mongooplog/mongoperf/morgorestore/mongosniff, etc.
Essentially mongodb-clients but guaranteed to work with the version of
mongo we have running. Is that silly?

Looking here: http://archive.ubuntu.com/ubuntu/pool/universe/m/mongodb/

It looks like the major size issue is the v8 stuff given that
mongodb-client on 2.2.4 is 20M, but on 2.4.6 it is 60M.

Is there a reason it can't be a shared library between the ~4 client
binaries?

Given that mongodb-server is 9.2M, I assume most of the size is those
extra client libs.

If it is just about what we are actively advocating having available
'mongorestore' would also go into the above list, but it is the sort
of thing that you don't do "regularly" in a healthy system.

John
=:->