I am working on a project in which we want to move a number of constantly running 'rules' from our main SQL databases onto a new system designed specifically to process these items in a VERY fast manner.
What happens right now is we get in requests from vendors, which we then need to run these requests through a series of rules to make sure they are good requests. Currently it is done through a mixture of T-SQL and scripts on MSSQL Databases. Each one of our vendors will normally submit 400-1000 of these requests a month, each request averages about 20 lines, and each of those lines we need to run against these rules.
If we get in 700 requests, averaging 20 lines per request, from 10 vendors and it takes a second to process each one it would take ~39 hours to do all 10 vendors requests.
So what I am looking to try do is get each one of these processed faster than that. On the hardware side we are looking at moving this system over to SSD based drives. But on the software I am wondering if this would be an appropriate system to look into a noSQL type model.
The main caveat is that we would be looking for something running on Windows Server. I know that there are many types of different noSQL systems and I honestly am not sure what would be appropriate, this is my first delve into that environment.
A few other pieces of information that would probably be helpful:
The rules which are running would be looking at the data in the noSQL system. The data would have information such as First Name, Last Name, Date1, Date2, and a few other key datapoints. The rules would look at the requests and compare them to the information found, then make sure it is a valid request. The scripts are mostly if/then statements at the moment.
As the data itself is all stored in SQL, we aren't really concerned with redundancy within the noSQL system itself - if there was an issue we would just repopulate that main database, and rerun the items through it.
Thank you ahead of time for anyone who is able to help.
What happens right now is we get in requests from vendors, which we then need to run these requests through a series of rules to make sure they are good requests. Currently it is done through a mixture of T-SQL and scripts on MSSQL Databases. Each one of our vendors will normally submit 400-1000 of these requests a month, each request averages about 20 lines, and each of those lines we need to run against these rules.
If we get in 700 requests, averaging 20 lines per request, from 10 vendors and it takes a second to process each one it would take ~39 hours to do all 10 vendors requests.
So what I am looking to try do is get each one of these processed faster than that. On the hardware side we are looking at moving this system over to SSD based drives. But on the software I am wondering if this would be an appropriate system to look into a noSQL type model.
The main caveat is that we would be looking for something running on Windows Server. I know that there are many types of different noSQL systems and I honestly am not sure what would be appropriate, this is my first delve into that environment.
A few other pieces of information that would probably be helpful:
The rules which are running would be looking at the data in the noSQL system. The data would have information such as First Name, Last Name, Date1, Date2, and a few other key datapoints. The rules would look at the requests and compare them to the information found, then make sure it is a valid request. The scripts are mostly if/then statements at the moment.
As the data itself is all stored in SQL, we aren't really concerned with redundancy within the noSQL system itself - if there was an issue we would just repopulate that main database, and rerun the items through it.
Thank you ahead of time for anyone who is able to help.