ssh tunnel not working

qinghai (support@nosqlbooster.com) ●

I studied this question, the answer is likely "the behavior is by design".

1. According to MongoDB Server selection spec.

Clients use the hostnames listed in the replica set config, not the seed list

This spec requires clients to connect to the hostnames listed in the ismaster response. Furthermore, if the response is from a primary, the client MUST remove all hostnames not listed. In this case, the client disconnects from "host_alias" and tries "host1" and "host2". (See updateRSFromPrimary.)Thus, replica set members must be reachable from the client by the hostnames listed in the replica set config.

2. In MongoDB Official Issue tracking system

Issue: Connect to ReplicaSet through SSH tunneling, the issue status is closed.

if you are willing to connect directly to a single known member of the set, you can do this by using one of the MongoClient constructors that takes a single ServerAddress instead of a List<ServerAddress>. These constructors make direct connections to servers and don't attempt any discovery of the other replica set members, which is what's getting you into trouble.

Another option, of course, is to set up a VPN, but I assume that's not an option for you.

3. MongoDB NodeJS driver

MongoDB Nodejs 2.1 or below uses the seed list, you can simply connect MongoDB Replicaset over SSH like this.

//localhost:30001 local forward to the remote server  REMOTE_HOST:20017
MongoClient.connect("mongodb://localhost:30001/test?replicaSet=rs", function(err, db) {
console.log("db connected")
});

Run this sample with driver 2.1, I got "db connected".

But If run the same code with MongoDB Nodejs Driver 2.2 or plus. I got the following error:

connect error { MongoError: failed to connect to server [127.0.0.1:27017] on first connect [MongoNetworkError: connect ETIMEDOUT ]

NoSQLBooster is depending on mongodb nodejs driver 2.2, we plan to upgrade to 3.0 once the official 3.0 driver is released.

I studied this question, the answer is likely "the behavior is by design".

1. According to MongoDB Server selection spec.

Clients use the hostnames listed in the replica set config, not the seed list

This spec requires clients to connect to the hostnames listed in the ismaster response. Furthermore, if the response is from a primary, the client MUST remove all hostnames not listed. In this case, the client disconnects from "host_alias" and tries "host1" and "host2". (See updateRSFromPrimary.)Thus, replica set members must be reachable from the client by the hostnames listed in the replica set config.

2. In MongoDB Official Issue tracking system

Issue: Connect to ReplicaSet through SSH tunneling, the issue status is closed.

if you are willing to connect directly to a single known member of the set, you can do this by using one of the MongoClient constructors that takes a single ServerAddress instead of a List<ServerAddress>. These constructors make direct connections to servers and don't attempt any discovery of the other replica set members, which is what's getting you into trouble.

Another option, of course, is to set up a VPN, but I assume that's not an option for you.

3. MongoDB NodeJS driver

MongoDB Nodejs 2.1 or below uses the seed list, you can simply connect MongoDB Replicaset over SSH like this.

//localhost:30001 local forward to the remote server  REMOTE_HOST:20017
MongoClient.connect("mongodb://localhost:30001/test?replicaSet=rs", function(err, db) {
console.log("db connected")
});

Run this sample with driver 2.1, I got "db connected".

But If run the same code with MongoDB Nodejs Driver 2.2 or plus. I got the following error:

connect error { MongoError: failed to connect to server [127.0.0.1:27017] on first connect [MongoNetworkError: connect ETIMEDOUT ]

NoSQLBooster is depending on mongodb nodejs driver 2.2, we plan to upgrade to 3.0 once the official 3.0 driver is released.

1

qinghai (support@nosqlbooster.com) ●

Thank you for your feedback.

If possible, could you please give me an accessible test MongoDB URI and SSH connection to allow me to recall the issue quickly?

Reply URL

1

qinghai (support@nosqlbooster.com) ●

We just released 3.5.2. The version upgrades the Node.js MongoDB driver to the latest. Could you please download and give it a try?

https://www.mongobooster.com/downloads

Reply URL

1

Vincent Chen ●

I have the same issue to connect to MongoDB Atlas replica set via SSH tunnel.

My mongobooster version is v3.5.6 free edition and MongoDB is Version 3.4.4

you may apply a free account in MongoDB Atlas to reproduce.

Here is the error message:

{
	"message" : "no primary found in replicaset",
	"stack" : "MongoError: no primary found in replicaset" +
              "at C:\\Users\\my_user\\AppData\\Local\\mongobooster\\app-3.5.6\\resources\\app.asar\\node_modules\\mongodb-core\\lib\\topologies\\replset.js:555:28" +
              "at Server.<anonymous> (C:\\Users\\my_user\\AppData\\Local\\mongobooster\\app-3.5.6\\resources\\app.asar\\node_modules\\mongodb-core\\lib\\topologies\\replset.js:312:24)" +
              "at Object.onceWrapper (events.js:290:19)" +
              "at emitOne (events.js:96:13)" +
              "at Server.emit (events.js:188:7)" +
              "at C:\\Users\\my_user\\AppData\\Local\\mongobooster\\app-3.5.6\\resources\\app.asar\\node_modules\\mongodb-core\\lib\\topologies\\server.js:299:14" +
              "at C:\\Users\\my_user\\AppData\\Local\\mongobooster\\app-3.5.6\\resources\\app.asar\\node_modules\\mongodb-core\\lib\\connection\\pool.js:469:18" +
              "at _combinedTickCallback (internal/process/next_tick.js:67:7)" +
              "at process._tickCallback (internal/process/next_tick.js:98:9)",
	"name" : "MongoError"
}

I have the same issue to connect to MongoDB Atlas replica set via SSH tunnel.

My mongobooster version is v3.5.6 free edition and MongoDB is Version 3.4.4

you may apply a free account in MongoDB Atlas to reproduce.

Here is the error message:

{
	"message" : "no primary found in replicaset",
	"stack" : "MongoError: no primary found in replicaset" +
              "at C:\\Users\\my_user\\AppData\\Local\\mongobooster\\app-3.5.6\\resources\\app.asar\\node_modules\\mongodb-core\\lib\\topologies\\replset.js:555:28" +
              "at Server.<anonymous> (C:\\Users\\my_user\\AppData\\Local\\mongobooster\\app-3.5.6\\resources\\app.asar\\node_modules\\mongodb-core\\lib\\topologies\\replset.js:312:24)" +
              "at Object.onceWrapper (events.js:290:19)" +
              "at emitOne (events.js:96:13)" +
              "at Server.emit (events.js:188:7)" +
              "at C:\\Users\\my_user\\AppData\\Local\\mongobooster\\app-3.5.6\\resources\\app.asar\\node_modules\\mongodb-core\\lib\\topologies\\server.js:299:14" +
              "at C:\\Users\\my_user\\AppData\\Local\\mongobooster\\app-3.5.6\\resources\\app.asar\\node_modules\\mongodb-core\\lib\\connection\\pool.js:469:18" +
              "at _combinedTickCallback (internal/process/next_tick.js:67:7)" +
              "at process._tickCallback (internal/process/next_tick.js:98:9)",
	"name" : "MongoError"
}

Reply URL

1

Vincent Chen ●

sorry I noticed this case is closed so I create new post:

http://feedback.mongobooster.com/topic/cannot-connect-to-mongodb-atlas-replica-set-via-ssh-tunnel

URL

1

qinghai (support@nosqlbooster.com) ●

I tried it with my test replica set+SSH key, It works well.

I hope to solve this issue, but can not reproduce it. If possible, could you please give me your test MongoDB replica set and test SSH key? One or two days is enough.

Reply URL

1

Pavel S ●

in my problem 10.0.1.206:27017 is arbiter

Reply URL

1

Pavel S ●

Cannot Reproduce - its very easy

21adcac8fc0ffcb65746e5327f9f662c 53f1d950c642e0f3f7cf12ab3a8ddf6c 91d960642d7f299a598691711ae2c761

Reply URL

1

qinghai (support@nosqlbooster.com) ●

Thank you for your feedback.

It seems that "discovery members" method is successful. Can you show me the specific connect error message?

And, Can you connect your test replica set server with mongo command-line + ssh tunnel (port forwarding . ssh -L 9000:localhost:27017user@example.com) ?

If the connection is successful, could you please show me the complete mongo command line?

URL

1

Pavel S ●

i open vpn + ssh - and connection opened.

you can see - what connection go with out ssh to servers.

mongodb 3.4.4

Reply URL

1

Dmitry Mikhaylov ●

I believe the problem is that after doing initial discovery using SSH tunnel, mongobooster tries to connect to individual replica set members directly (i.e. not using SSH tunnel). The expected behavior is that mongobooster will use SSH tunnel to connect to discovered replica set member.

This is based on netstat logs Pavel has provided:

There are direct connections from local IP address (192.168.0.216) to mongodb port on private IP address of 10.0.1.206. As I understand, 10.0.1.x network is normally not accessible from outside (that is why SSH tunnel is needed) and connections work only because Pavel has set-up VPN tunnel.
So if mongobooster would work as expected, we would not see those connections, but instead would see another SSH connection to bastion host (54.XXX.143.166)

To reproduce the problem w/o having remote subnet, you would need following:

a replica set (maybe even 1-member replica set would be enough), say with IPs rs1, rs2, rs3
a "bastion" host with sshd (with different IP address), with IP bh
configure mongobooster to connect to replica set via SSH tunnel to "bastion" host
Connect
Find mongobooster PID, mpid
Use netstat to see which address mongobooster has used to actually connect to replica set member: netstat -p | fgrep <mpid>
Intended behaviour: mongobooster has connections only to "bastion" host IP bh
What will most likely happen: mongobooster will have direct connection to some replica set member

Reply URL

1

qinghai (support@nosqlbooster.com) ●

Thanks for your very detailed answer, I can reproduce the issue now.

Suppose I already have a 3-nodes replica set in the remote host (e.g example.com). I can connect the replica set in the remote machine with the following mongo command line.

mongo "mongodb://127.0.0.1:20001,127.0.0.1:20002,127.0.0.1:20003/?readPreference=primary&replicaSet=rs"

Now, I want to connect it locally, I try to set up SSH tunnel like this:

ssh -L 30001:127.0.0.1:20001 -L 30002:127.0.0.1:20002 -L 30003:127.0.0.1:20003 user@example.com

And connect it use mongo command line

mongo "mongodb://127.0.0.1:30001,127.0.0.1:30002,127.0.0.1:30003/?readPreference=primary&replicaSet=rs"

But no luck, I got the following error:

2017-12-19T21:04:39.718+0800 I NETWORK Starting new replica set monitor for rs/127.0.0.1:30001,127.0.0.1:30002

2017-12-19T21:04:39.990+0800 I NETWORK Successfully connected to 127.0.0.1:30002 (1 connections now open to 127.0.0.1:30002 with a 5 second timeout)

2017-12-19T21:04:39.990+0800 I NETWORK [ReplicaSetMonitor-TaskExecutor-0] Successfully connected to 127.0.0.1:30001 (1 connections now open to 127.0.0.1:30001 with a 5 second timeout)

2017-12-19T21:04:40.147+0800 I NETWORK [ReplicaSetMonitor-TaskExecutor-0] changing hosts to rs/127.0.0.1:20001,127.0.0.1:20002 from rs/127.0.0.1:30001,127.0.0.1:30002

2017-12-19T21:04:45.146+0800 W NETWORK [ReplicaSetMonitor-TaskExecutor-0] Failed to connect to 127.0.0.1:20001 after 5000ms milliseconds, giving up.

2017-12-19T21:04:45.146+0800 W NETWORK Failed to connect to 127.0.0.1:20002 after 5000ms milliseconds, giving up.

2017-12-19T21:04:50.651+0800 W NETWORK Failed to connect to 127.0.0.1:20001 after 5000ms milliseconds, giving up.

2017-12-19T21:04:52.418+0800 E - Error saving history file: FileOpenFailed: Unable to fopen() file

2017-12-19T21:04:52.418+0800 I CONTROL shutting down with code:0

I would like to ask, how to connect replica set over SSH tunnel by using common SSH command and mongo command?

Thanks a lot

Thanks for your very detailed answer, I can reproduce the issue now.

Suppose I already have a 3-nodes replica set in the remote host (e.g example.com). I can connect the replica set in the remote machine with the following mongo command line.

mongo "mongodb://127.0.0.1:20001,127.0.0.1:20002,127.0.0.1:20003/?readPreference=primary&replicaSet=rs"

Now, I want to connect it locally, I try to set up SSH tunnel like this:

ssh -L 30001:127.0.0.1:20001 -L 30002:127.0.0.1:20002 -L 30003:127.0.0.1:20003 user@example.com