Tag: mongodb

MongoDB: Changing Host in Replica Set

When we get replacement servers at work, they frequently build a new server with a temporary name and IP address with the plan of swapping the host name and IP with the decommed server. So my Server123 gets turned down, Server123-Temp gets renamed to Server123, and the IP from the old server is configured on the replacement. Everything is operating exactly as it was before even if the direct host name or IP address were used — great for not needing to update firewall rules and vpn profiles, but I encountered a bit of a problem with the MongoDB cluster.

When I initiated the replica set, I did not have to specify a host name. It pulled the host name from the system — which happily provided that temporary name that doesn’t really exist (it’s not in DNS). Which was fine — I could add the temporary name to /etc/hosts along with the future name that I’ve remapped to the current IP so my “new” VMs all talk to each other and the old servers don’t get mucked up.

But, eventually, I’d like the replica set to have the right names. Had I known about this ahead of time, I’d simply have changed the host name value on the box to be the permanent name, initialized the replica set, and returned the temporary name to the box. But I didn’t, and I didn’t really want to start from 0 with the database I’d restored. Luckily, it turns out there’s a process for re-creating the replica set without destroying the data.

First, edit the mongo.conf file and comment out the replica set block. Restart each server with the new config. Then delete the “local” database from each MongoDB server using mongo local --eval "db.dropDatabase()"

Uncomment the replica set block in mongo.conf and restart MongoDB again — initialize the replica set again (make sure the server “knows” it’s proper name first!)

MongoDB: Increasing Log Level

We had a problem with an application accessing our MongoDB cluster, and the log files didn’t provide much useful information. I could see the client connect and disconnect … but nothing in between. I discovered that the default logging level is very low. Good for disk utilization and I/O, but not great for troubleshooting.

db.runCommand({getParameter: 1, logLevel: 1}) # Get the current logging level
db.setLogLevel(3) # Fairly robust logging
db.setLogLevel(5) # don't try this is prod huge stream of text super logging
db.setLogLevel(0) # and set logging back to a low level once you are done troubleshooting

You can also increase the log level for individual components of MongoDB to minimize logging I/O:

db.setLogLevel(2, "accessControl" )

 

MongoDB: Setting Up a Replica Set

On one server create a key file. Copy this key file to all other servers that will participate in the replica set

mkdir -p /opt/mongodb/keys/
openssl rand -base64 756 > /opt/mongodb/keys/$(date '+%Y-%m-%d').key
chmod 400 /opt/mongodb/keys/$(date '+%Y-%m-%d').key
chown -R mongodb:mongodb /opt/mongodb/keys/$(date '+%Y-%m-%d').key

On each server, edit /etc/mongo.conf and add the keyfile to the security section and define a replica set

security:
 authorization: enabled
 keyFile:  /etc/mongodb/keys/mongo-key
#replication:
replication:
  replSetName: "myReplicaSet"

Restart MongoDB on each node.

On one server, use mongosh to enter the MongDB shell.

rs.initiate(
{
_id: "myReplicaSet",
members: [
{ _id: 0, host: "mongohost1.example.net" },
{ _id: 1, host: "mongohost2.example.net" },
{ _id: 2, host: "mongohost3.example.net" }
]
})

Use rs.status() to view the status of the replica set. If it is stuck in STARTING … check connectivity. If the port is open, I ran into a snag with some replacement servers. They’ve got temporary hostnames. But you cannot add a host on itself — it ignores that you typed mongohost1.example.net … and it takes it’s hostname value. And then sends that value to the other servers in the replica set. If you cannot change the hostname to match what you want, there is a process to change the hostname in a replicaset.

MongoDB: Where is my shell?!?

We are upgrading servers from really old MongoDB (4.2.15) to much new MongoDB (6.something). I am used to getting into the MongoDB shell using:

mongoserver:~ # mongo -u $STRMONGOUSER -p $STRMONGOPASS
MongoDB shell version v4.2.15
connecting to: mongodb://127.0.0.1:27017/?compressors=disabled&gssapiServiceName=mongodb
Implicit session: session { "id" : UUID("5658a72f-fea0-4316-aa97-4b0c0ffab7ff") }
MongoDB server version: 4.2.15

Except the new server says there’s no such file. And there isn’t. A bit of research later, I learn that the shell is now called mongosh … which is a more reasonable name. It works the same way: mongosh -u $STRMONGOUSER -p $STRMONGOPASS gets me there, and all of the commands I know work.

Backing up (and restoring) *All* Data in MongoDB

The documentation on Mongo’s website tells you to use mongodump with a username, password, destination, and which database you want to back up. Except I wanted to back up and restore everything. Users, multiple databases, I don’t really know what else is in there hence I want everything instead of enumerating the things I want.

Turns out you can just omit the database name and it dumps everything

mongodump --uri="mongodb://<host URL/IP>:<Port>" -u $STRMONGODBUSER -p $STRMONGODBPASS

And restore with

mongorestore --uri="mongodb://<host URL/IP>:<Port>"

Since it’s a blank slate with no authentication or users defined yet.

MongoDB: Basics

We inherited a system that uses MongoDB, and I managed to get the sandbox online without actually learning anything about Mongo. The other environments, though, have data people care about set up in a replicated cluster of database servers. That seems like the sort of thing that’s going to require knowing more than “it’s a NoSQL database of some sort”.

It is a NoSQL database — documents are organized into ‘collections’ within the database. You can have multiple databases hosted on a server, too. A document is a group of key/value pairs with dynamic schema (i.e. you can just make up keys as you go).

There are GUI clients and a command-line shell … of course I’m going with the shell 🙂 There is a db function for basic CRUD operations using db.nameOfCollection then the operation type:

db.collectionName.insert({"key1": "string1", "key2" : false, "key3": 12345})
db.collectionName.find({key3 : {$gt : 10000} })
db.collectionName.update({key1 : "string1"}, {$set: {key3: 100}})
db.collectionName.remove({key1: "string1"});

CRUD operations can also be performed with NodeJS code — create a file with the script you want to run, then run “node myfile.js”

Create a document in a collection

var objMongoClient = require('mongodb').MongoClient;
var strMongoDBURI = "mongodb://mongodb.example.com:27017/";
  
objMongoClient.connect(strMongoDBURI, function(err, db) {
  if (err) throw err;
    var dbo = db.db("dbNameToSelect");
    var objRecord = { key1: "String Value1", key2: false };
    dbo.collection("collectionName").insertOne(objRecord, function(err, res) {
         if (err) throw err;
         console.log("document inserted");
         db.close();
    });
}); 

Read a document in a collection

var objMongoClient = require('mongodb').MongoClient;
var strMongoDBURI = "mongodb://mongodb.example.com:27017/";

objMongoClient.connect(strMongoDBURI, function(err, db) {
  if (err) throw err;
    var dbo = db.db("dbNameToSelect");
    var objQuery = { key1: "String Value 1" };
    dbo.collection("collectionName").find(objQuery).toArray(function(err, result) {
     if (err) throw err;
     console.log(result);
     db.close();
  });
}); 

Update a document in a collection

var objMongoClient = require('mongodb').MongoClient;
var strMongoDBURI = "mongodb://mongodb.example.com:27017/";

objMongoClient.connect(strMongoDBURI, function(err, db) {
if (err) throw err;
  var dbo = db.db("dbNameToSelect");
  var objRecord= { key1: "String Value 1" };
  dbo.collection("collectionName").deleteOne(objRecord, function(err, obj) {
    if (err) throw err;
    console.log("Record deleted");
    db.close();
});
}); 

Delete a document in a collection

var objMongoClient = require('mongodb').MongoClient;
var strMongoDBURI = "mongodb://mongodb.example.com:27017/";

objMongoClient.connect(strMongoDBURI, function(err, db) {
if (err) throw err;
  var dbo = db.db("dbNameToSelect");
  var objQuery = { key1: "String Value 1" };
  var objNewValues = { $set: {key3: 12345, key4: "Another string value" } };
  dbo.collection("collectionName").updateOne(objQuery, objNewValues , function(err, res) {
    if (err) throw err;
    console.log("Record updated");
    db.close();
   });
}); 

OpenHAB Cloud Installation Prerequisites

We started setting up the OpenHAB cloud server locally, and the instructions we had found omitted a few important steps. They say ‘install redis’ and ‘install mongodb’ without providing any sort of post-install configuration.

Redis
# This is optional – if you don’t set a password, you’ll just get a warning on launch that a password was supplied but none is required. While the service is, by default, bound to localhost … I still put a password on everything just to be safe

vi /etc/redis.conf # Your path may vary, this is Fedora. I've seen /etc/redis/redis.conf too

# Find the requirepass line and make one with your password

480 # requirepass foobared
requirepass Y0|_|RP@s5w0rdG03s|-|3re

# Restart redis

service redis restart

Mongo:
# Install mongo (dnf install mongo mongo-server)
# start mongodb

service mongod start

# launch mongo client

mongo

# Create user in admin database

db.createUser({user: "yourDBUser", pwd: "yourDBUserPassword", roles: [{role: userAdminAnyDatabase", db: "admin"}]});
exit

# Modify mongodb server config to use security

vi /etc/mongod.conf

# remove remarkes before ‘security: ‘ and ‘authorization’ – set authorization to enabled:

99 # secutiry Options - Authorization and other security settings
100 security:
101 # Private key for cluster authentication
102 #keyFile: <string>
103
104 # Run with/without security (enabled|disabled, disabled by default)
105 authorization: enabled

# restart mongo

service mongod restart

#Launch mongo client supplying username and connecting to the admin database

mongo -uyourDBUser -p admin

# it will connect and prompt for password – you can use db.getUser to verify the account (but you just logged into it, so that’s a bit redundant)

MongoDB shell version: 3.2.12
Enter password:
connecting to: admin
> > db.getUser("yourDBUser");
{
        "_id" : "admin.yourDBUser",
        "user" : "yourDBUser",
        "db" : "admin",
        "roles" : [
                {
                        "role" : "userAdminAnyDatabase",
                        "db" : "admin"
                }
        ]
}

# Create the openhab database — mongo is a bit odd in that “use dbname” will switch context to that database if it exists *and* create the databse if it doesn’t exist. Bad for typo-prone types!

use yourDBName;

# Create the user in the openhab database

db.createUser({user: "yourDBUser", pwd: "yourDBUserPassword", roles: [{role: readWrite", db: "yourDBName"}]});

# You can use get user to verify it works

db.getUser("yourDBUser");
exit

# Now you can launch the mongo client connecting to the openhab database:

mongo -uyourDBUser -p yourDBName

# It will prompt for password and connect. At this point, you can use “node app.js” to launch the openhab cloud connector. Provided yourDBUser, yourDBUserPassword, and yourDBName match what you’ve used in the config file … it’ll connect and create a bunch of stuff