Make SQL Monitor scale to more machines and networks
CONTEXT: We want to monitor many different servers in different domains and locations
PROBLEM: Network configuration for cross-domain monitoring can be challenging, and SQL Monior can slow down if many servers are being mointored
• Create a central web interface that can access multiple Base monitors
• Provide an optional agent that can be deployed for easier network configuration
• Make a single installation of SQL Monitor scale to a larger number of servers, and make network configuration easier
Thanks for your feedback Don.
We definitely intend to look at the install and upgrade process, so we'll do our best to address (1) and (2) in the next batch of work on this feature.
3-5 look like bugs - we've added these to our tracking system - thanks again for bringing these to our attention.
Don Ferguson commented
It works well. I appreciate the implementation of this.
A few areas where it could be improved are:
1. With a multiple base monitor configurations, when rolling out upgrades to one monitor, it will take everything down until all is upgraded. This underscored the need for a more efficient upgrade process.
2. It is possible to have multiple web servers and base monitors that cross each other. That's a good thing, but setting it up required manually editing the RedGate Response AuthorizedClients.config file and adding the hex string values from other installs into the file. That process could be better defined and possibly built into a setup dialog.
3. When changing Alias configurations, they don't automatically propagate to the other web server installs and they need to be reset to see the changes.
4. When sorting overview page on severity, I would expect to see the red instances on top, followed by the yellow, then green. That does not seem to be the case.
5. The formatting of the Group and Severity Sort dialogs does not display correctly in Internet Explorer; however, it looks fine in Chrome.
Don Ferguson commented
The ability to scale out in a large SQL Server installation environment is an improvement that is desperately needed. While there have been performance improvements in some of the later 5.x releases, multiple monitor installations are still needed to get halfway decent performance when monitoring hundreds of SQL Servers. The problem is that with all of these installations, it makes it difficult to work out of each separate interface. So having a way to bring it altogether in a single interface would help a great deal.
"Something" really needs to be done to make this a whole lot easier. I've been a SQL Monitor user since the first v3 came out and am one of those Accidental DBAs who have a love-hate relationship with all things Windows and SQL Server security/Login/User stuff.
Having just reset a Windows box at Rackspace and using SQL Monitor on Azure, it took a half day to get SQL Monitor and that box to agree to being monitored.
Having had to erase everything to with the box's history from SQL Monitor and re-add it, rather than any re-deploy Action, I'm just glad that adding a box to SQLM isn't a daily chore/endurance/pain-in-the-posterior.
Even having read the SQLM documentation re Security, and the prior non-DBA's write-up (proved inaccurate), scaling to more than a few boxes is a limitation/weakness of the current SQLM iteration (5.1 BTW).
The far-end installer needs help to make it usable by A-DBA's like me.
The Host needs help with re-deployment. We absolutely need Friendly Name overrides on EVERY SINGLE PAGE - so tired of seeing a Group Name (friendly), then get stuck with an I.P. or a Rackspace device-ID on emails/subpages/configs at random... Even things like that will promote scalability by making the tool consistent/easier to use. It's often a case of "do I have to go over to SQLM to.....?" Sorry, but that's the truth...
Sherry Lehr commented
Just wondering how long before 5.1 is out.
eric twilegar commented
I think each server should just have a locally installed agent. Having to open ports like 135 between the base on the server is weird to say the least.
eric twilegar commented
I'd prefer installing an agent on each sql server that pipes data back to a central basemonitor/web access stack. The amount of ports that have to be open between the base monitor and your sql server is disturbing.
I'd love to see this move forward. Especially with the capacity to automatically discover new servers as they come online within a specified network.
Paul Millar commented
Create a client agent (we monitor from our domain, across VPNs to customer sites/datacenters - opening RPC/WMI/et al does not go down well, nor does fixing RPC to individual port ranges.
It can be done, but large customers / hosting providers pretty much make life a lot harder than it needs to be, distracting majorly from the main monitoring / support / diagnosis tasks - an agent with single or few port requirements would make life so much easier - assuming LAN-like line of site doesn't work well for us. Deploy agent with a service account addresses one of the domain trust suggestions too.
I would like this also. My work network mandates distinct VLANs / Domains for each group of servers, heavilly firewalled and with no trust on the domains.
so using Windows Authentication is impossible, as is fixing the WMI ports on monitored servers. having 2 or more "central management" is not exactly "central" anymore. having multiple servers capturing data on distinct networks and reporting to one single database would be great.
I was also part of the beta for SQL Monitor Hosted, and as an example this could work for that product also: having multiple relays could make our life easier when we need to monitor servers on distinct domains/vlans.
Maximiliano Morales commented
el proceso RedGate.Response.Engine.Alerting.Base.Service.exe consume mas de 5gb de ram, firstname.lastname@example.org
seria bueno limitar el consumo de ram
Be sure to consider time differences. Untrusted domains may not have the same time source and therefore not be in sync. We are having problems with this very issue.
Eric Humphrey commented
I'd like to add that sometimes you need to use different base monitors to monitor different subnets or vlans of servers, but all report back into the same repository. The whole goal is one pane of glass for all servers, not a segmented view.
Look at the way Paessler PRTG handles this. Allow a web service to hook to multiple base monitors. PRTG calls them Probes. The web service just connects to the base monitors via ports so Admins could punch holes in firewalls to make this work. Users can login to the web server either with native or Windows accounts (hint hint) and see databases in any variety of situations domains, etc.
This would be quite helpful. I know that our environment is an edge case but we're having trouble trying to figure out how we can monitor 1200+ databases across several instances without setting up multiple base monitors and web servers. Maybe another title for this could be a "Scale out" base monitor solution?
Priya Sinha commented
Thanks a lot for clarifying.
I would like to view all the base monitors at the same time, from one location.
Priya Sinha commented
Thanks a lot nhartman for this suggestion.
Would you like to view all the base monitor at the same time? Or is the problem only associated with switching between web servers?
Given the sizes of some SQL environments we found that it is necessary to have multiple base monitor servers. However currently it is cumbersome to have to navigate to multiple URL's or switch the base monitor that's tied to a web server. Having a central console that is easily and always connected to multiple base monitors would alleviate this problem. Also this could allow you to monitor from various networks and domains.
This is something we are considering. We have some ideas of how we could achieve this, for example a remote monitor (which can be installed on external network and will collect data and send to base monitor) but nothing is concrete yet.
Please let us know what considerations we would need to make for your own environments.