Given the recent news about Microsoft's Tay chatbot being pulled off the internet after being exploited for its vulnerabilities, I was wondering if there is much research out there on the value of chatbots as part of a 'conversational interface' design trend, and whether there are specific applications that are more useful or popular compared to others. Slack and similar software applications have been using it for a while, but in so far as replacing actual humans online (because most of the time there doesn't seem to be anyone there) so that there is always someone/something you can talk to, there only seems to be a financial/resource gain for the company (minus the cost of implementation and maintenance).
Also, I was wondering if there are any documented processes or procedures for testing chatbots with actual end users, and if this testing is more quantitative due to the nature of the chatbots or if there are some qualitative assessments that are also done as part of the testing/commissioning process.
UPDATE: Facebook has just recently launched their Messenger Platform, which allows businesses and developers to access their chatbot API to build applications that incorporate this technology. No doubt Facebook has accumulated enough user data to come up with their design and concept, although how they used the data to create their design might remain a mystery.