Manage NordVPN recommended servers through addons API page

[Jack Yaz]

Good point! Fallback checking could be done in the "getRecommended" function though so it wouldn't be a big change. I would start with "country+city" (if set) and fallback to "country only" on fail (and if set) and if that also fails fallback to the default recommendation ! Obviously, however the config is stored for the UI and how the UI automatically gets and displays the list of available countries and cities adds to the complexity of the solution.

This gets the recommended server for USA...
curl -s -m 5 "https://api.nordvpn.com/v1/servers/recommendations?filters\[servers_groups\]\[identifier\]=legacy_standard&filters\[servers_technologies\]\[identifier\]=openvpn_udp&filters\[country_id\]=228&limit=1"

Had to use country_id (228)! This gets all countries and cities with ids (that I don't know how to add to the filter on the "recommended" api):
curl -s -m 5 "https://api.nordvpn.com/v1/servers/countries" | jq --raw-output '.[] | . as $parent | .cities[] | [$parent.name, $parent.id, .name, .id] | "\(.[0]) id is \(.[1]) - \(.[2]) id is \(.[3])"'

The "city" MUST be filtered in the api call and not in the jq processing to be a true "recommendation" from NordVPN. Unless as @Jack Yaz suggests that we write our own check.

If the script could let you choose a city too - it would provide a greater capability than NordVPNs own "recommended server" page

City display is easy, we could do an hourly (probably daily is better) download of countries and ids and store them in a file. I can then display that in the webui as a select dropdown. We'd need an extra setting per client for country id, and whether to use a country for the recommendation or not.

 

[Jack Yaz]

Or i play with fetch/xmlhttprequest and do it all in the UI...

 

[h0me5k1n]

I'll let you decide ! Does it really need to be updated often - once selected, a user is rarely going to change the configuration as that's the whole point of the script (to do the work for them). I'd probably just have an "Update Locations" action/button to refresh the list of available countries and cities - would this work? Would the UI pick this up straight away or would the page need to be refreshed?

With regards to the script logic... the process may look like this?

1. If a city and country is selected, pull all the possible servers available and find the ones with the lowest load (top 2 maybe?) and then ping test them for the quickest response and use the one that comes out on top.
2. If a country is selected, use the recommended api with a filter to find the server to use
3. Otherwise just get the basic recommended server.

Does this sound about right?

 

[Marin]

I cannot thank you guys enough for pursuing this! I hope I didn’t create more work for you!


Sent from my iPhone using Tapatalk

 

[Jack Yaz]

I'll let you decide ! Does it really need to be updated often - once selected, a user is rarely going to change the configuration as that's the whole point of the script (to do the work for them). I'd probably just have an "Update Locations" action/button to refresh the list of available countries and cities - would this work? Would the UI pick this up straight away or would the page need to be refreshed?

With regards to the script logic... the process may look like this?

1. If a city and country is selected, pull all the possible servers available and find the ones with the lowest load (top 2 maybe?) and then ping test them for the quickest response and use the one that comes out on top.
2. If a country is selected, use the recommended api with a filter to find the server to use
3. Otherwise just get the basic recommended server.

Does this sound about right?

if we only keep the list of countries and cities in memory, we can query it on demand and no page refresh needed. this approach would also work for the cli menu. i guess the program flow for point 1 is:

1) generate a list of countries and cities, with ids -> user selects their preference

curl -s -m 5 "https://api.nordvpn.com/v1/servers/countries?limit=16384" | jq --raw-output '.[] | . as $parent | .cities[] | [$parent.name, $parent.id, .name, .id] | "[\"\(.[0])\",\"\(.[1])\",\"\(.[2])\",\"\(.[3])\"]"'

2) use selected country as a filter for recommendation api, get top 5 and sort by load (querying all servers and filtering in jq is VERY SLOW on the router (tested on my 86U)

curl --silent "https://api.nordvpn.com/v1/servers/recommendations?filters\[country_id\]=227&limit=5" | jq --raw-output ' .[] | select(.locations[].country.city.id="2989907")' | jq --raw-output --slurp ' sort_by(.load) | limit(5;.[]) | [
.hostname, .load] | "\(.[0]): \(.[1])"'

3) 5s (is that sufficient) ping test to each of the 5 servers,
4) sort servers by avg ping in ascending order
5) configure VPN to use lowest average ping server

does that look right?

 

[Chris0815]

@Jack Yaz and @h0me5k1n, not sure if you are aware of this script (https://github.com/jotyGill/openpyn-nordvpn), but it may go in the direction to select servers of NordVPN from different locations. Maybe there is something you can use...?
I never was able to get in run on my end... But you are the coders that can make things working for people like me that are not so familiar with command lines and these kind of stuff...
Thank you again for the script as it is already today!!!

 

[Jack Yaz]

@Jack Yaz and @h0me5k1n, not sure if you are aware of this script (https://github.com/jotyGill/openpyn-nordvpn), but it may go in the direction to select servers of NordVPN from different locations. Maybe there is something you can use...?
I never was able to get in run on my end... But you are the coders that can make things working for people like me that are not so familiar with command lines and these kind of stuff...
Thank you again for the script as it is already today!!!

i've seen that script, but python is a big overheard imo - we're doing rather well with our simple yet effective implementation

 

[Chris0815]

i've seen that script, but python is a big overheard imo - we're doing rather well with our simple yet effective implementation

Sure! Sorry for not knowing the differences that come along with it. Simplicity is highly appreciated!

 

[h0me5k1n]

if we only keep the list of countries and cities in memory, we can query it on demand and no page refresh needed. this approach would also work for the cli menu. i guess the program flow for point 1 is:

1) generate a list of countries and cities, with ids -> user selects their preference

curl -s -m 5 "https://api.nordvpn.com/v1/servers/countries?limit=16384" | jq --raw-output '.[] | . as $parent | .cities[] | [$parent.name, $parent.id, .name, .id] | "[\"\(.[0])\",\"\(.[1])\",\"\(.[2])\",\"\(.[3])\"]"'

2) use selected country as a filter for recommendation api, get top 5 and sort by load (querying all servers and filtering in jq is VERY SLOW on the router (tested on my 86U)

curl --silent "https://api.nordvpn.com/v1/servers/recommendations?filters\[country_id\]=227&limit=5" | jq --raw-output ' .[] | select(.locations[].country.city.id="2989907")' | jq --raw-output --slurp ' sort_by(.load) | limit(5;.[]) | [
.hostname, .load] | "\(.[0]): \(.[1])"'

3) 5s (is that sufficient) ping test to each of the 5 servers,
4) sort servers by avg ping in ascending order
5) configure VPN to use lowest average ping server

does that look right?

Yes! Looks right to me!

It might be worth leaving in the development branch for a while once this is done so that it can be tested over a few days. I'd like to test side by side and see how the server chosen differs from the pure recommendation to country and the pseudo one... Probably by writing to a log saying "country and city recommended server is X - country recommended server is Y and pure recommended server is Z"

 

[kid63]

Hi all,

first thx for the nice script, workin' flawlessly, second jq doesn't survive a reboot, can i call it from init-start?

Running on RT-AX88U

cu
kid

 

[Jack Yaz]

Hi all,

first thx for the nice script, workin' flawlessly, second jq doesn't survive a reboot, can i call it from init-start?

Running on RT-AX88U

cu
kid

jq is installed via entware, perhaps we need a check to wait for entware to come up before running anything

 

[kid63]

Thx for the answer Jack , would be great to implement this and thx in advance

Yeah, i know it's entware, but it's an important part. I've tried your cmds for the country and city also, just perfect. I'm really excited to see, where this script will end up.

cu

 

[Jack Yaz]

Yes! Looks right to me!

It might be worth leaving in the development branch for a while once this is done so that it can be tested over a few days. I'd like to test side by side and see how the server chosen differs from the pure recommendation to country and the pseudo one... Probably by writing to a log saying "country and city recommended server is X - country recommended server is Y and pure recommended server is Z"

sure, i can do that. i'll start by adding settings to store country and city choice

 

[Jack Yaz]

Thx for the answer Jack , would be great to implement this and thx in advance

Yeah, i know it's entware, but it's an important part. I've tried your cmds for the country and city also, just perfect. I'm really excited to see, where this script will end up.

cu

https://github.com/jackyaz/nvpnmgr/commit/b6034fbbf0e29b3bc73b6d30d66fbf9230ee8fb2
patched to check for ntp and entware. in practise you shouldnt trip this unless you open nvpnmgr really quickly after a reboot!

 

[Jack Yaz]

@h0me5k1n how's this? not used in the recommendation api query yet! wanted feedback on the country selection
https://github.com/jackyaz/nvpnmgr/commit/487e3315a918054579f97e76fae61f7c7e5987a9

 

[kid63]

https://github.com/jackyaz/nvpnmgr/commit/b6034fbbf0e29b3bc73b6d30d66fbf9230ee8fb2
patched to check for ntp and entware. in practise you shouldnt trip this unless you open nvpnmgr really quickly after a reboot!

Wow, that was lightning fast, thx a lot Jack.

Edit: Countryselection works perfect too, nothing more to say!

cu

 

[Jack Yaz]

City prompting done too. Currently all the menus do is store the selected country and city name and ids in the config file.

Tomorrow I'll work on having the "get recommended" taking into account chosen country and city.

 

[h0me5k1n]

@h0me5k1n how's this? not used in the recommendation api query yet! wanted feedback on the country selection
https://github.com/jackyaz/nvpnmgr/commit/487e3315a918054579f97e76fae61f7c7e5987a9

Looking good! It's funny how the country list is long enough that you actually have to scroll back up to see the ones at the top - the joys of having lots of countries to choose from ... but this is in a console anyway so it doesn't really matter! Once it's visible in the UI, I doubt there will be a need to run it from the command line after the initial installation!

City prompting done too. Currently all the menus do is store the selected country and city name and ids in the config file.

Tomorrow I'll work on having the "get recommended" taking into account chosen country and city.

Damn! That was quick! I only just had enough time to test that commit and you've already done the next!

 

[Jack Yaz]

Looking good! It's funny how the country list is long enough that you actually have to scroll back up to see the ones at the top - the joys of having lots of countries to choose from ... but this is in a console anyway so it doesn't really matter! Once it's visible in the UI, I doubt there will be a need to run it from the command line after the initial installation!



Damn! That was quick! I only just had enough time to test that commit and you've already done the next!

the webui won't be too bad - i'll just feed the country data and city data into arrays and have them populate select boxes of some description. when you click a country, you'll see the cities for that country

 

[h0me5k1n]

City prompting done too.

Also looking great! There's even logic to know which countries have a single location or multiple cities to choose from!

A question @Jack Yaz ... what does "managed" mean?