Sunday, June 30, 2019

Using Azure DevOps to collaborate Data Scientists and App Developers


What is your most favorite IDE? A developer can mention various of IDE’s. How about data scientist?


Normally, the Developers develop the code and data scientists train develop and train the model. Tools such as VSCode come in handy as easy to install and use. Many have many preferences so the chances are that it can be a choice of the developer or the development team. Usually the trained model is handed over to the app developer for integrating it and build the final application. There are times where the mismatches in compatibility can cost both the app developer and the model developer. The resulting friction between app developers and data scientists to identify and fix the root cause can be a slow, frustrating, and expensive process
We often here organizations including managers continuously talking about Artificial Intelligence. People like to find solutions that are integrated with AI. So as the developers have a development lifecycle, the data scientists follow a data science lifecycle. 

The lifecycle includes processes such as,

Data Ingestion --> Data Preparation --> Model Development --> Model Deployment

There can be many iterations of this lifecycle as there can be requirements for changing the data labels, removing anomalies, changes upon user feedback and timely decision changes and many more. 
Application Development Lifecycle for a developer includes building app, testing, maintaining, and continuously adopt to changes in user requirements where the models of data scientists also can be a part of. There shall be a good understanding between the application developers and model developers as mismatches can lead to errors in the integrated systems. I have seen many issues with even traditional SSRS implementations to web applications. When the model changes, app fails. 

Azure DevOps and Azure Machine Learning are combined to overcome this issue with a solution suggested by Microsoft as shown in the image below. 

Source: Azure Blog

Both the app developers and model developers need to use Git as the repository for managing the development artifacts. 

If we simplify this process to understand, the model training code developed by the data scientist will trigger the Azure DevOps CI/CD Pipeline as available on the right top of the diagram above. There the developer can execute multiple steps such as unit tests, integration tests, training and etc.. The changes made by the app developer will trigger integration tests whenever a code change is pushed to the repository. There can be triggers set on data lake as well to execute different commands. The Model Store will register the new models and trigger releases for human approval followed by many other necessary tests as shown in the bottom right. These procedures will help both the developers coordinate and work well within the changes required as well. 
The detailed article on Azure Blog can be found here.

55 comments:

  1. This implies the labyrinth has no out of reach segments, no roundabout ways, no open territories.ExcelR Data Science Courses

    ReplyDelete
  2. Very informative post! There is a lot of information here that can help any business get started with a successful social networking campaign. Admond Lee

    ReplyDelete
  3. I admire this article for the well-researched content and excellent wording. I got so involved in this material that I couldn’t stop reading. I am impressed with your work and skill. Thank you so much. anaconda install tensorflow

    ReplyDelete
  4. I am looking for and I love to post a comment that "The content of your post is awesome" Great work!

    Digital marketing course

    ReplyDelete
  5. Thanks for the informative article. This is one of the best resources I have found in quite some time. Nicely written and great info. I really cannot thank you enough for sharing.

    AWS Online Training
    AWS Certification Training
    AWS Certification Course Online
    AWS Training
    AWS Online Course

    ReplyDelete
  6. I recently found many useful information in your website especially this blog page. Among the lots of comments on your articles. Thanks for sharing. apps

    ReplyDelete
  7. This is very significant, and yet necessary towards just click this unique backlink: northshore connect to success

    ReplyDelete
  8. Wow, cool post. I'd like to write like this too - taking time and real hard work to make a great article... but I put things off too much and never seem to get started. Thanks though. Tableau Data Blending

    ReplyDelete
  9. Thanks for the valuable information and insights you have so provided here... buy instagram likes cheap paypal

    ReplyDelete
  10. I am really enjoying reading your well written articles. It looks like you spend a lot of effort and time on your blog. I have bookmarked it and I am looking forward to reading new articles. Keep up the good work.

    data analytics courses hyderabad
    data science training
    business analytics course

    ReplyDelete
  11. Awesome blog. I enjoyed reading your articles. This is truly a great read for me. I have bookmarked it and I am looking forward to reading new articles. Keep up the good work!


    business analytics course
    data science course in hyderabad
    data analytics courses hyderabad

    ReplyDelete
  12. I like viewing web sites which comprehend the price of delivering the excellent useful resource free of charge. I truly adored reading your posting. Thank you!
    data science training in guwahati

    ReplyDelete
  13. It is the kind of information I have been trying to find. Thank you for writing this information. It has proved utmost beneficial for me. faça o download do Appvn apk 2020

    ReplyDelete
  14. I really like your take on the issue. I now have a clear idea on what this matter is all about.. acesse aqui para baixar Lucky Patcher 2020

    ReplyDelete
  15. That is really nice to hear. thank you for the update and good luck. blackmart alpha 0 49 b 93 apk

    ReplyDelete
  16. This is such a great resource that you are providing and you give it away for free. I love seeing blog that understand the value of providing a quality resource for free. visit site

    ReplyDelete
  17. Utilize the @mentions highlight of Twitter that will assist you with building more grounded connections and draw in a more focused on crowd's consideration.how to grow twitter followers

    ReplyDelete
  18. Took me time to understand all of the comments, but I seriously enjoyed the write-up. It proved being really helpful to me and Im positive to all of the commenters right here! Its constantly nice when you can not only be informed, but also entertained! I am certain you had enjoyable writing this write-up. uk iptv

    ReplyDelete
  19. Thanks for the blog loaded with so many information. Stopping by your blog helped me to get what I was looking for. text message marketing for small business

    ReplyDelete
  20. Enjoy millions of the latest Android apps, games, music, movies, TV, books, magazines & more. Apps

    ReplyDelete
  21. I’m going to read this. I’ll be sure to come back. thanks for sharing. and also This article gives the light in which we can observe the reality. this is very nice one and gives indepth information. thanks for this nice article... hikedatabase.com/united-states/hiking-in-rhode-island/

    ReplyDelete
  22. Maintaining: The Science maintains the processed data using data warehousing, data cleansing, data staging, and data architecture. data science course in hyderabad

    ReplyDelete
  23. I am looking for and I love to post a comment that "The content of your post is awesome" Great work! data science courses

    ReplyDelete
  24. Can’t tell a good article, because it is more than that, thanks keep going...
    PLACEMENT TRAINING

    ReplyDelete
  25. This can be achieved by meeting all those who are going to be involved with the project. As we all know, consulting is a business which involves interaction and close coordination, so find the best output for your business by selecting the ideal Salesforce Consulting Company. Salesforce training in Hyderabad

    ReplyDelete
  26. This is why it is better that you should linked review before developing. It is possible to post superior send that way. App Development

    ReplyDelete
  27. The Master's Guide to text marketing services
    A large number of private companies use instant message advertising to empower expanded promoting commitment from possibilities and clients. As far as we can tell, most independent ventures use text showcasing to produce expanded client securing, convey advancements and warnings, augment client support viability, and increment inside productivity.
    From retail organizations and not-for-profit associations to land and eateries, text showcasing is utilized by more independent companies than at any time in recent memory.

    ReplyDelete
  28. Excellent Blog! I would like to thank for the efforts you have made in writing this post. I am hoping the same best work from you in the future as well. I wanted to thank you for this websites! Thanks for sharing. Great websites!

    data science course in India

    ReplyDelete
  29. Its a great pleasure reading your post.Its full of information I am looking for and I love to post a comment that "The content of your post is awesome" Great work. Log Cost CRM

    ReplyDelete
  30. You delivered such an impressive piece to read, giving every subject enlightenment for us to gain information. Thanks for sharing such information with us due to which my several concepts have been cleared. Legacy data archiving

    ReplyDelete
  31. Very good points you wrote here..Great stuff...I think you've made some truly interesting points.Keep up the good work.data science course in Hyderabad

    ReplyDelete
  32. Texting is an extremely effective communication & marketing channel for tax preparation services. 98% of text messages are read within 3 minutes of receipt! That’s significant when you consider that 90% of emails are considered SPAM so email marketing for tax professionals just isn’t as effective anymore. Whether you are marketing an accounting firm or just practice marketer

    ReplyDelete
  33. I want to say thank you so much and sharing this excellent article about DevOps Consulting Services.

    ReplyDelete
  34. Notice:Publishers are free to use this article on an ezine or website provided the article is reprinted in its entirety, including copyright and disclaimer, and ALL links remain intact and active. data science course in india

    ReplyDelete
  35. Textellent offers a texting service specially designed for the TaxPro. It works directly with all the leading software for tax preparers allowing you to provide exceptional and personal customer service while supporting you in marketing tax services as well tax preparation software

    ReplyDelete
  36. You've covered all points very clearly I was looking for. Thank you for sharing useful information.

    ReplyDelete
  37. I've read this post and if I could I desire to suggest you some interesting things or suggestions. Perhaps you could write next articles referring to this article. I want to read more things about it!
    Data Science courses

    ReplyDelete
  38. Thank you for your posting about aws blogs. It is useful to freshers who start to get knowledge about the aws. I got basic idea with the help of your details..
    https://www.kellytechno.com/Hyderabad/Course/amazon-web-services-training

    ReplyDelete
  39. Online lottery, Thai lottery, Malaysian lottery, Vietnamese lottery is available for you to play all. End every need, good price, get a lot, less waste. There are more than 100 slot games, fun, สมัคร ufa, easy to play, beautiful images, beautiful graphics, fast bonus if you want to get money quickly Must play slots

    ReplyDelete
  40. สล็อตออนไลน์ (Slot Online) คือการนำเครื่องเล่นพนัน ตู้สล็อตแมชชีน ที่กล่าวมาข้างต้นนี้ นำมาทำเป็นเกมอิเล็กทรอนิกส์ที่เรียกกันว่า สล็อตออนไลน์ เนื่องด้วยยุคสมัยพัฒนาคนหันมานิยมเล่นพนันกันผ่านคอมพิวเตอร์ จะได้นำเกมสล็อตมาทำเป็นเกมพนันออนไลน์ ผ่านระบบเครื่อข่ายอินเตอร์เน็ต ซึ่งผู้เล่นสามารถเล่นแบบผ่าน โปรแกรมสล็อต ก็ได้หรือจะเป็นการเข้าเล่น Slots ผ่านหน้าเว็บไซต์ผู้ให้บริการ ซึ่งบริการเกม สล็อตออนไลน์ นั้นก็มีรูปแบบของกติกาการเล่น คล้ายกันกับการเล่นบนตู้สล็อตแมชชีน ทั้งภาพและเสียงสมจริงเร้าใจไม่แพ้กันกับไปนั่งเล่นในคาสิโนเลยทีเดียว.

    ReplyDelete
  41. ufabet Parent company, the most popular online gambling website, whether it is online football betting Online casinos Baccarat online All of them were not less popular than each other. Become a member of UEFA Bet Playing with the parent company Did not pass agent Bet on a variety of casino games Especially the gambler who likes to Online football betting Our website provides 4 football odds, football betting, a minimum of 10 baht , betting is easy

    ReplyDelete
  42. Great to become visiting your weblog once more, it has been a very long time for me. Pleasantly this article i've been sat tight fosuch a long time. I will require this post to add up to my task in the school, and it has identical subject along with your review. Much appreciated, great offer. data science course in nagpurr

    ReplyDelete
  43. Generally excellent review. I absolutely love this site. Much appreciated!
    best interiors

    ReplyDelete
  44. Brilliant Blog! I might want to thank you for the endeavors you have made recorded as a hard copy of this post. I am trusting a similar best work from you later on also. I needed to thank you for these sites! Much obliged for sharing. Incredible sites!
    data science training institute in hyderabad

    ReplyDelete
  45. Really good information and useful content. Thanks for sharing with us. Keep doing update blogs more.
    Data Science Certification in Hyderabad

    ReplyDelete
  46. I just couldn't leave your website before telling you that I truly enjoyed the top quality info you present to your visitors? Will be back again frequently to check up on new posts. business analytics course in kanpur

    ReplyDelete
  47. Excellent effort to make this blog more wonderful and attractive.
    data science classes in hyderabad

    ReplyDelete
  48. The Digi uprise Course in Rajouri Garden most popular course "Foundations," teaches students the basic principles that comprise The Digi uprise Framework. It helps them identify the market, listen to clients, and decrease the possibility of missing out on potential opportunities.

    ReplyDelete
  49. origynivf is proud and satisfied to have an outstanding success rate and excellence under her supervision. Origyn IVF center is one of the India’s largest IVF center/Test Tube Baby Center in North India having highly advanced equipments & IVF techniques It is an Initiative to originate new life, through the best team effort.

    ReplyDelete
  50. The
    Best Dietitian and Nutritionist in Delhi - Diet Mantra by Monika
    by Monika is focused on providing expert nutritional advice to suit your body needs and empowering you in making lasting lifestyle changes. One of the biggest challenges one faces in today’s fast-paced life is how to actually keep up with the time crunch and how to actually follow a healthy lifestyle. Taking the right regimen for good wellbeing is significant and according to the various maladies and diseases, it is critical to have a solid eating routine with the goal that the immunity system could help to defend the infection and can help to improve your health. We offer online diet consultation so our customers get handily associated with us with no hustle. Monika Manchanda has been awarded as the best dietitian & nutritionist in Delhi.

    ReplyDelete
  51. Baby Joy IVF is the Best IVF Centre in Delhi, well-known for its expertise and efficient results. Baby Joy IVF center is led by a team of dedicated and experienced professionals and IVF experts, ensuring that you receive the best medical advice and service. We specialize in customized minimal stimulation (mini-IVF), natural cycles (natural IVF), and conventional IVF protocols tailored to the needs of each individual. At Baby Joy IVF Clinic in India, we provide world-class fertility treatments by using cutting-edge technology and the experience of leading gynecologists, IVF specialist team, and resourceful physicians to create results-oriented and cost-effective plans to take advantage of ideal outcomes and provide the Highest IVF Success Rate for treatments in a safe and supportive environment.

    ReplyDelete