Generative Models

Disclaimer: These are just notes I'll use for references for future projects, or if any is interested to see my notes on Generative Models

 What is a generative model?
A generative model, is a model for randomly generated data, aka observable data. When given some hidden parameters, it can create a probability distribution over observation and label sequences, Some cases a distribution can be created from a generative model through Baye's rule.
Baye's rule 
In probability theory and statistics, Bayes' theorem(alternatively Bayes' law or Bayes' rule) describes the probability of an event, based on prior knowledge of conditions that might be related to the event. 
Source: https://en.wikipedia.org/wiki/Bayes%27_theorem
There are a few types of Generative models (Source: https://en.wikipedia.org/wiki/Generative_model)



  • Gaussian mixture model and other types of mixture model
  • Hidden Markov model
  • Probabilistic context-free grammar
  • Naive Bayes
  • Averaged one-dependence estimators
  • Latent Dirichlet allocation
  • Restricted Boltzmann machine
  • Generative adversarial networks


    • Training a generative adversarial model
    After some research about the best way to train a model is not to train a model that assigns high likelihood to test data OR even require the model to learn without using any labels. High likelihood is too consistent and it can learn any distribution if given infinite data, and a perfect model class. However, when using finite data it has the possibility to produce models that can "overgeneralize". The whole process can be found here:
    http://www.inference.vc/how-to-train-your-generative-models-why-generative-adversarial-networks-work-so-well-2/


    This document is incomplete, Will add or correct more information in the future as I progressively research and understand Generative Models, Do not rely on this post for "Reliable information" rather take a look at the provided sources for more detailed and correct information about Generative Models. 

    Comments

    Post a Comment

    Popular posts from this blog

    The fall of Cesar

    Image
    Introduction   Cesar is a “hacker” who is “known” on Kik for pretending to be someone that he isn’t, what I mean by this is that he goes around pretending that he is a hacker, and claims to be a white hat just because he “exposes pedophiles”, The interesting part that he does not just expose pedophiles but rather he abuses his trust with his friends to target people he despises. One of our main victims would be his ex which we will name “Stacey” just to protect her privacy. Stacey was 16 at the time she was dating Cesar, it was believed that Cesar was 15 or 16, we do not know because his age often changes because he tells a different story to everyone. Stacey was dating Cesar until she became 18 when she decided to end the relationship with Cesar simply because she did not feel like she wanted to continue to be in a long distance, online relationship and wishes to seek something that’s more physical and close to home. Cesar did not like this idea one bit so at the time of ...
    Read more

    Reading Comprehension Systems

    Image
    While browsing articles related to Machine Learning, I have found something quite interesting from the Computer Science Department of Stanford University called " Evaluating Reading Comprehension Systems ". The following information compiled in this post is information taken from reliable sources which I will provide links to if you are interested in reading the full version. This post is simply meant to explain what " Reading Comprehension Systems " are in a simple matter. What is Reading Comprehension Systems? Reading Comprehension, also known as RC are systems designed to understand a given text and return answers as a response to questions about the text. Example: (Source:  https://nlp.stanford.edu/pubs/jia2017adversarial.pdf ) When you give the system this Article Article:  Super Bowl 50   Paragraph:  “ Peyton Manning became the first quarterback ever to lead two different teams to multiple Super Bowls. He is also the oldest quarterback ever to pl...
    Read more