I want to write a proposal on one of the MONGODB issues! I chose a problem that it uses high memory for data storage., is that correct?

Stack Exchange Network


Stack Exchange network consists of 176 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

discord.js – Discord My mongoDB keeps disconnecting multiple times

Every time I use a command that uses MongoDB when it finishes, it’s set to disconnect from mongo. For some reason, every time I finish the command, MongoDB disconnects an extra time. See the attachment I provided below.mongoDBconnectionerr. As you can see in the image, every time a command is run, mongo DB connects and disconnects 1 extra time. This is my code.

        if(args(0) === 'welcomemsg') {

            var text = args.splice(2).join(' ');
            var channelidfrommessage = args(1);

            if (message.content.includes('<#') && message.content.includes('>')) {
                //message = message.toString().slice(-5)
                var channelIDSTRIP = message.toString().replace(/(^d.-)/g, '');
                //message = message.toString().slice(2, -1);
                console.log(channelIDSTRIP);
                console.log('channelname detected');

                
            await mongo().then(async mongoose => {
                try {
                   await welcomeSchema.findOneAndUpdate(
                       {
                           _id: message.guild.id
                       }, {
                           _id: message.guild.id,
                           channelID: channelIDSTRIP,
                           text: text
                       }, {
                           upsert: true,
                       }
                   );
               } finally {
                   mongoose.connection.close();
               }
           });

            } else {
                const actualchannel = message.guild.channels.cache.get(channelidfrommessage) || message.guild.channels.cache.find((channel) => channel.name === channelidfrommessage);

                
            await mongo().then(async mongoose => {
                try {
                   await welcomeSchema.findOneAndUpdate(
                       {
                           _id: message.guild.id
                       }, {
                           _id: message.guild.id,
                           channelID: actualchannel,
                           text: text
                       }, {
                           upsert: true,
                       }
                   );
               } finally {
                   mongoose.connection.close();
               }

mongodb – Which NoSQL for storing images as 2D arrays (BLOBs)

I have large number of images that I need to access on per-pixel-basis in my app. The images are downloaded from web and are quite large (4k). Update is done several times a day (tousands of images), reading is non-stop.

Currently I use PostGis and store images tiled in raster column type. However, performance is not the best, mainly updates are quite slow (transaction, index update etc).

So I would like to use NoSQL database, since I really dont need SQL.

My workflow is:

Insertion – download image -> split it to tiles of size 64×64 (compress them if necessary with arithetic or huffman coder) -> store tiles in database

Selection – read pixel (x, y) -> locate tile -> retrieve tile -> “unpack” tile in memory -> locate pixel (x, y) -> return value to user

What database should I use for this purpose? I have looked to MongoDB and Cassandra. The later one seems more appropriate for this task.

mongodb – Notification for host failure and health status

I set up a mongodb cluster with replica sets for high availability. My configuration seems to work well, as the database can operate even if I turn off some of my hosts.

My question is: How can I find out that a host failed or get the health status of the cluster? I wish to have something like a dashboard or a “health status”. My current solution is to connect to the cluster and query the replication state once in a while.

Can anybody recommend a tool which reports failover per email or provides a kind of dashboard where I can see the cluster health?

I already searched for solutions and found solutions like this:
Percona Monitoring and Management but it considers performance metrics rather than failover related information.

Thank you for your help.

MongoDB 4.2 Shard Cluster: what is db.cache.chunks.DB and how to flush it

Here is my MongoDB cluster (I didn’t draw about Mongos but exist) I made “news_cat” db and created some collections and I did sharding.

I delete DB (“news_cat”) in Mongos using db.dropDatabase() but in the replica set2 in config DB
“news_cat”, which I had already deleted, exists also in collection too.

db.cache.chunks

rstest02:PRIMARY> db.
Display all 3234 possibilities? (y or n)
db.adminCommand()
db.cache.chunks.news_cat.col1
db.cache.chunks.news_cat.col2
db.cache.chunks.news_cat.col3
...

and I also checked about log in config and too many logs exist caused by “news_cat”:

   2020-09-08T14:25:29.566+0000 I  SH_REFR  (ConfigServerCatalogCacheLoader-7) Refresh for database news_cat took 0 ms and failed :: cau
sed by :: NamespaceNotFound: database news_cat not found

https://jira.mongodb.org/browse/SERVER-50241 also same symptom.

I thought if there’s some issue about dropDatabase() command I checked in the documentation https://docs.mongodb.com/v4.2/reference/method/db.dropDatabase/

I did flushRouterConfig but it doesn’t work.

mongodb replication failed with socketexception and listener

I’m trying to set up mongodb replication using two servers in different data center. For the purposes of this question, I’ll say their IP address are 1.1.1.1 and 2.2.2.2. I installed a clean version of centos8 and mongo with these commands on each server:

yum install -y yum-utils
cat <<EOT >> /etc/yum.repos.d/mongodb-org-4.4.repo
(mongodb-org-4.4)
name=MongoDB Repository
baseurl=https://repo.mongodb.org/yum/redhat/8Server/mongodb-org/4.4/x86_64/
gpgcheck=1
enabled=1
gpgkey=https://www.mongodb.org/static/pgp/server-4.4.asc
EOT
yum install -y mongodb-org-4.4.1 mongodb-org-server-4.4.1 mongodb-org-shell-4.4.1 mongodb-org-mongos-4.4.1 mongodb-org-tools-4.4.1
mkdir -p /data/db
sed -i "s/bindIp:.*/bindIp: 0.0.0.0/g" /etc/mongod.conf
systemctl start mongod

This worked great. I can do ssh root@1.1.1.1 and then I can run mongo --host 2.2.2.2 to enter the mongo cli of server 2.2.2.2 from server 1.1.1.1. I can also ssh root@2.2.2.2 and run mongo --host 1.1.1.1 to enter the mongo cli of server 1.1.1.1 from server 2.2.2.2.

Next, I ssh root@1.1.1.1. Then I ran the command:

 mongod --auth --replSet "rs0" --bind_ip localhost,2.2.2.2

But this failed and gave this output:

(root@li1953-23 ~)# mongod --auth --replSet "rs0" --bind_ip localhost,2.2.2.2
{"t":{"$date":"2020-11-15T01:27:26.192+00:00"},"s":"I",  "c":"CONTROL",  "id":23285,   "ctx":"main","msg":"Automatically disabling TLS 1.0, to force-enable TLS 1.0 specify --sslDisabledProtocols 'none'"}
{"t":{"$date":"2020-11-15T01:27:26.197+00:00"},"s":"W",  "c":"ASIO",     "id":22601,   "ctx":"main","msg":"No TransportLayer configured during NetworkInterface startup"}
{"t":{"$date":"2020-11-15T01:27:26.198+00:00"},"s":"I",  "c":"NETWORK",  "id":4648601, "ctx":"main","msg":"Implicit TCP FastOpen unavailable. If TCP FastOpen is required, set tcpFastOpenServer, tcpFastOpenClient, and tcpFastOpenQueueSize."}
{"t":{"$date":"2020-11-15T01:27:26.198+00:00"},"s":"W",  "c":"ASIO",     "id":22601,   "ctx":"main","msg":"No TransportLayer configured during NetworkInterface startup"}
{"t":{"$date":"2020-11-15T01:27:26.199+00:00"},"s":"I",  "c":"STORAGE",  "id":4615611, "ctx":"initandlisten","msg":"MongoDB starting","attr":{"pid":130556,"port":27017,"dbPath":"/data/db","architecture":"64-bit","host":"li1953-23"}}
{"t":{"$date":"2020-11-15T01:27:26.199+00:00"},"s":"I",  "c":"CONTROL",  "id":23403,   "ctx":"initandlisten","msg":"Build Info","attr":{"buildInfo":{"version":"4.4.1","gitVersion":"ad91a93a5a31e175f5cbf8c69561e788bbc55ce1","openSSLVersion":"OpenSSL 1.1.1c FIPS  28 May 2019","modules":(),"allocator":"tcmalloc","environment":{"distmod":"rhel80","distarch":"x86_64","target_arch":"x86_64"}}}}
{"t":{"$date":"2020-11-15T01:27:26.199+00:00"},"s":"I",  "c":"CONTROL",  "id":51765,   "ctx":"initandlisten","msg":"Operating System","attr":{"os":{"name":"CentOS Linux release 8.2.2004 (Core) ","version":"Kernel 4.18.0-193.14.2.el8_2.x86_64"}}}
{"t":{"$date":"2020-11-15T01:27:26.199+00:00"},"s":"I",  "c":"CONTROL",  "id":21951,   "ctx":"initandlisten","msg":"Options set by command line","attr":{"options":{"net":{"bindIp":"localhost,2.2.2.2"},"replication":{"replSet":"rs0"},"security":{"authorization":"enabled"}}}}
{"t":{"$date":"2020-11-15T01:27:26.200+00:00"},"s":"E",  "c":"STORAGE",  "id":20568,   "ctx":"initandlisten","msg":"Error setting up listener","attr":{"error":{"code":9001,"codeName":"SocketException","errmsg":"Cannot assign requested address"}}}
{"t":{"$date":"2020-11-15T01:27:26.200+00:00"},"s":"I",  "c":"REPL",     "id":4784900, "ctx":"initandlisten","msg":"Stepping down the ReplicationCoordinator for shutdown","attr":{"waitTimeMillis":10000}}
{"t":{"$date":"2020-11-15T01:27:26.201+00:00"},"s":"I",  "c":"COMMAND",  "id":4784901, "ctx":"initandlisten","msg":"Shutting down the MirrorMaestro"}
{"t":{"$date":"2020-11-15T01:27:26.201+00:00"},"s":"I",  "c":"SHARDING", "id":4784902, "ctx":"initandlisten","msg":"Shutting down the WaitForMajorityService"}
{"t":{"$date":"2020-11-15T01:27:26.201+00:00"},"s":"I",  "c":"NETWORK",  "id":4784905, "ctx":"initandlisten","msg":"Shutting down the global connection pool"}
{"t":{"$date":"2020-11-15T01:27:26.201+00:00"},"s":"I",  "c":"REPL",     "id":4784907, "ctx":"initandlisten","msg":"Shutting down the replica set node executor"}
{"t":{"$date":"2020-11-15T01:27:26.201+00:00"},"s":"I",  "c":"NETWORK",  "id":4784918, "ctx":"initandlisten","msg":"Shutting down the ReplicaSetMonitor"}
{"t":{"$date":"2020-11-15T01:27:26.201+00:00"},"s":"I",  "c":"SHARDING", "id":4784921, "ctx":"initandlisten","msg":"Shutting down the MigrationUtilExecutor"}
{"t":{"$date":"2020-11-15T01:27:26.201+00:00"},"s":"I",  "c":"CONTROL",  "id":4784925, "ctx":"initandlisten","msg":"Shutting down free monitoring"}
{"t":{"$date":"2020-11-15T01:27:26.201+00:00"},"s":"I",  "c":"FTDC",     "id":4784926, "ctx":"initandlisten","msg":"Shutting down full-time data capture"}
{"t":{"$date":"2020-11-15T01:27:26.201+00:00"},"s":"I",  "c":"STORAGE",  "id":4784927, "ctx":"initandlisten","msg":"Shutting down the HealthLog"}
{"t":{"$date":"2020-11-15T01:27:26.201+00:00"},"s":"I",  "c":"STORAGE",  "id":4784929, "ctx":"initandlisten","msg":"Acquiring the global lock for shutdown"}
{"t":{"$date":"2020-11-15T01:27:26.201+00:00"},"s":"I",  "c":"-",        "id":4784931, "ctx":"initandlisten","msg":"Dropping the scope cache for shutdown"}
{"t":{"$date":"2020-11-15T01:27:26.201+00:00"},"s":"I",  "c":"CONTROL",  "id":20565,   "ctx":"initandlisten","msg":"Now exiting"}
{"t":{"$date":"2020-11-15T01:27:26.201+00:00"},"s":"I",  "c":"CONTROL",  "id":23138,   "ctx":"initandlisten","msg":"Shutting down","attr":{"exitCode":48}}

I am unable to continue with the mongodb tutorial on how to set up replication until I remedy this problem. What did I do wrong and how do I fix?


EXTRA

I also tried to ssh root@1.1.1.1 and then mongo to enter cli of mongo instance at 1.1.1.1. Then I ran this command:

rs.initiate( {
_id :  "rs0",
members: (
{ _id:  0, host:  "1.1.1.1:27017" },
{ _id:  1, host:  "2.2.2.2:27017" }
)
})

Which gives the error:

{
        "ok" : 0,
        "errmsg" : "This node was not started with the replSet option",
        "code" : 76,
        "codeName" : "NoReplicationEnabled"
}

mongodb – WiredTiger serves data. What exactly is the in-memory data?

I’m having a hard time understanding this paragraph

At the start of an operation, WiredTiger provides a point-in-time snapshot of the data to the operation. A snapshot presents a consistent view of the in-memory data.

  1. What is a point-in-time snapshot?

Maybe I’m confusing with the wording and any write operation is written to a this in-memory file/snapshot with a specific timestamp, sort of?

MongoDB Collection Level Permission Issues

I am trying to set up a MongoDB user that only has access to certain collections across several databases. The issue is that these databases are created dynamically at runtime.

I decided to employ collection level access in order to do this with the following permission on the user role:

"db" : "admin",
"privileges" : [ 
    {
        "resource " : {
            "cluster" : true
        },
        "actions" : [ 
            "listDatabases"
        ]
    }, 
    {
        "resource" : {
            "db" : "",
            "collection" : "packages"
        },
        "actions" : [ 
            "find", 
            "insert", 
            "remove", 
            "update", 
            "listCollections"
        ]
    }]

What I want is for any user with this role to be able to read/write/remove in any database but only on the packages collection inside of that database, and be unable to read or write to any other collection. When I launch mongod in –auth I do not see any databases besides admin.

I am certain this is just me being unable to figure out exactly what actions / resources / roles I am supposed to grant.