AI Grammar Question Solver

AI Grammar Question Solver — independent reviews, comparisons, pricing and step-by-step guides on Aizhi.

  • POP-11

    POP-11

    POP-11 is a reflective, incrementally compiled programming language with many of the features of an interpreted language. It is the core language of the Poplog programming environment developed originally by the University of Sussex, and recently in the School of Computer Science at the University of Birmingham, which hosts the main Poplog website. POP-11 is an evolution of the language POP-2, developed in Edinburgh University, and features an open stack model (like Forth, among others). It is mainly procedural, but supports declarative language constructs, including a pattern matcher, and is mostly used for research and teaching in artificial intelligence, although it has features sufficient for many other classes of problems. It is often used to introduce symbolic programming techniques to programmers of more conventional languages like Pascal, who find POP syntax more familiar than that of Lisp. One of POP-11's features is that it supports first-class functions. POP-11 is the core language of the Poplog system. The availability of the compiler and compiler subroutines at run-time (a requirement for incremental compiling) gives it the ability to support a far wider range of extensions (including run-time extensions, such as adding new data-types) than would be possible using only a macro facility. This made it possible for (optional) incremental compilers to be added for Prolog, Common Lisp and Standard ML, which could be added as required to support either mixed language development or development in the second language without using any POP-11 constructs. This made it possible for Poplog to be used by teachers, researchers, and developers who were interested in only one of the languages. The most successful product developed in POP-11 was the Clementine data mining system, developed by ISL. After SPSS bought ISL, they renamed Clementine to SPSS Modeler and decided to port it to C++ and Java, and eventually succeeded with great effort, and perhaps some loss of the flexibility provided by the use of an AI language. POP-11 was for a time available only as part of an expensive commercial package (Poplog), but since about 1999 it has been freely available as part of the open-source software version of Poplog, including various added packages and teaching libraries. An online version of ELIZA using POP-11 is available at Birmingham. At the University of Sussex, David Young used POP-11 in combination with C and Fortran to develop a suite of teaching and interactive development tools for image processing and vision, and has made them available in the Popvision extension to Poplog. == Simple code examples == Here is an example of a simple POP-11 program: define Double(Source) -> Result; Source2 -> Result; enddefine; Double(123) => That prints out: 246 This one includes some list processing: define RemoveElementsMatching(Element, Source) -> Result; lvars Index; [[% for Index in Source do unless Index = Element or Index matches Element then Index; endunless; endfor; %]] -> Result; enddefine; RemoveElementsMatching("the", [[the cat sat on the mat]]) => ;;; outputs [[cat sat on mat]] RemoveElementsMatching("the", [[the cat] [sat on] the mat]) => ;;; outputs [[the cat] [sat on] mat] RemoveElementsMatching([[= cat]], [[the cat]] is a [[big cat]]) => ;;; outputs [[is a]] Examples using the POP-11 pattern matcher, which makes it relatively easy for students to learn to develop sophisticated list-processing programs without having to treat patterns as tree structures accessed by 'head' and 'tail' functions (CAR and CDR in Lisp), can be found in the online introductory tutorial. The matcher is at the heart of the SimAgent (sim_agent) toolkit. Some of the powerful features of the toolkit, such as linking pattern variables to inline code variables, would have been very difficult to implement without the incremental compiler facilities.

    Read more →
  • Top 10 AI Text-to-video Tools Compared (2026)

    Top 10 AI Text-to-video Tools Compared (2026)

    Trying to pick the best AI text-to-video tool? An AI text-to-video tool is software that uses machine learning to help you get more done — it scales effortlessly from a single task to thousands. The best picks balance beginner-friendly simplicity with the depth power users need, and they ship updates often. Whether you are a beginner or a pro, the right AI text-to-video tool slots into your workflow and pays for itself fast. This guide breaks down the top picks, their pros and cons, and who each one is best for.

    Read more →
  • How to Choose an AI Copywriting Tool

    How to Choose an AI Copywriting Tool

    Trying to pick the best AI copywriting tool? An AI copywriting tool is software that uses machine learning to help you get more done — it scales effortlessly from a single task to thousands. The best picks balance beginner-friendly simplicity with the depth power users need, and they ship updates often. Whether you are a beginner or a pro, the right AI copywriting tool slots into your workflow and pays for itself fast. Read on for hands-on impressions, pricing tiers, and the standout features that matter.

    Read more →
  • MedSLT

    MedSLT

    MedSLT is a medium-ranged open source spoken language translator developed by the University of Geneva. It is funded by the Swiss National Science Foundation. The system has been designed for the medical domain. It currently covers the doctor-patient diagnosis dialogues for the domains of headache, chest and abdominal pain in English, French, Japanese, Spanish, Catalan and Arabic. The vocabulary used ranges from 350 to 1000 words depending on the domain and language pair. == Motivation for creating MedSLT == With more than 6000 languages worldwide, language barriers become an increasing problem for healthcare. The lack of medical interpreters can lead to disastrous consequences. These range from prolonged hospital stays to wrong diagnosis and medication. A study found that only about half of the 23 million people with limited proficiency in English in the United States had been provided with a medical interpreter. Millions of refugees and immigrants worldwide face similar problems, although not always as severe. The gap between need and availability of language services might be closed with speech translation systems. == Challenges == The biggest challenge is and was to develop an ideal system, though it is not possible to do so at this moment. This system would fit the needs of doctors and the patients alike, and would provide accurate and flexible translation. A realisation of an ideal translation tool is impossible without the use of unrestricted language and a large vocabulary. Medical professionals demand high reliability from translation. This favours rule-based architectures over data-driven. The latter are more suitable for inexperienced users. Rule-based architectures achieve higher accuracy especially if used by experts. Though it is highly desirable to build a bidirectional system supporting a two-way dialogue, which concentrates on patient-centered communication, the patients will have difficult access to the system. Most patients have no experience with such systems. Less reliable results for translation from the patient-to-doctor direction are the outcome. To overcome this the system needs to provide either easy access or an integrated help tool to guide the users through the process. Although controlled rule-based systems achieve good results, they are brittle. To receive good translations the user needs to be familiar with the system and has to know what is covered by the grammar. Covering different sub-domains (headache, chest and abdominal pain) and language pairs presents additional problems. A shared structure and grammar for all subdomains and language pairs minimises development and maintenance costs. The integration of new doctor and patient languages is also a key challenge. Adding new languages should be quick and rather simple, because he system has to be used in many countries to cover multiple language pairs. Direct translation from source to target language proves to be rather difficult. Using interlingua for unidirectional translation instead of a bidirectional approach helps to simplify the translation process. On top of this, the system has to run on different platforms, because mobility is a key issue for many attending physicians. A portable version addresses these issues, but has to deal with the heavy load of the translation process. == The MedSLT system == The system's speech recognition is based on the Nuance 8.5 platform that supports grammar-based language models. All grammars used for recognition, analysis and generation are compiled from a small set of unification grammars. These core grammars are created by the open-source Regulus Grammar Compiler and are automatically specialised using corpus-driven methods. The specialisation considers both the task (recognition, analysis and generation) and the sub-domain (headache, chest and abdominal pain). The specialisation uses the explanation-based learning algorithm to create a treebank from the training corpus. These examples are divided into sets of subtrees by using domain- and grammar-specific rules (also known as "operationality criteria" in machine translation). The subtree rules are combined into a single rule, creating a specialised unification grammar. The grammar is compiled to an executable form, for analysis and generation by a parser or generator, and for recognition of a CFG grammar. A CFG grammar is required for the Nuance engine. Compilation by Nuance-specific criteria turns the grammar into speech recognition packages. The final step uses the training corpus again for statistical tuning of the language model. MedSLT translation processes are based on a rule-based interlingua. The interlingua is treated as an actual language (it is a very simple version of English) and is specified by a Regulus grammar. This grammar does not take account of complex surface syntax phenomena of real languages like movement or agreement. A set of rules is the base for translating the source language semantic representation to interlingua. Another set of rules covers the translation from interlingua to the target language. The semantic representations are converted to surface words using a target language grammar. Defining semantics for a specific domain enables the developers to specify interlingua with a small, tightly constraint semantic grammar. The translations based on interlingua match direct translations almost perfectly, because the development shifts to a decoupled monolingual architecture. A set of combined interlingua corpora, with one corpus per sub-domain, is the core of this architecture. All source language development corpora are translated to interlingua. These are sorted and grouped together with the corresponding source language examples. The interlingua forms are then translated into each target language, and the results are attached together. This organisation improves the translation process. There is no duplicated effort for multilingual regression testing, because each parsing and generation step is performed once. This allows more frequent testing. The representation language used for all forms is Almost Flat Functional semantics. AFF is derived from the Spoken Language Translator, the precursor of MEdSLT. SLT uses Quasi Logical Form, a logical based representation language. QLF is an expressive yet very complex language, causing high development and maintenance costs. A minimal solution was planned for the medical translator. Early versions of the system utilised a language using simple feature-value lists. These lists were supplemented with an optional level of nesting to represent subordinate clauses (i.e. embedded clauses). Determiners were not included, because they are hard to translate and it is difficult to reliably distinguish and recognise them. This way, translation rules became a lot simpler, because only a list of feature-value pairs had to be mapped to another list of pairs. The language turned out to be underconstrained. Adding natural sortal constraints to the grammar solved this problem, but also returned the language to a more expressive formalism. The newly created AFF combines elements of QLF and the feature-value list semantics. This version of flat semantics is enhanced with additional functional markings. This together with a relatively small vocabulary solved the ambiguity problem of the original flat representation language without creating overly complex rules. In addition, the syntactic structures are treated carefully by a compromise of linguistic and engineering traditions. The grammars are in fact retrieved from linguistically motivated resource, using corpus-based methods. They are driven by small sets of examples. This results in simpler and flatter domain-specific grammars. The semantics are less sophisticated and represent a minimal approach in the engineering tradition. Each lexical item contributes a set of feature-value pairs. This leads to simple-to-write translation rules. There are only lists of features-value pairs to map to other feature-value pairs. However, as a result the machine translation channel model becomes underspecified and is weakened, whereas the target language model is strengthened. An intelligent help module is integrated into the system to support users in utilising the full coverage of the grammars. This tool provides the user with examples as close as possible to the users original utterance. The output is based on a library. Each sub-domain and language pair has its own library. The contents are extracted from the combined interlingua corpora. The help module scans the corpus for the tagged source language form mapped with the corresponding target language form. Additionally a second statistical recogniser is used as backup. The results are used to select similar examples from the library. According to the generation preferences, one of the derived strings is picked and the target language string is realised as spoken language. Some statistical corpus based meth

    Read more →
  • Find It, Fix It

    Find It, Fix It

    Find It, Fix It is a mobile app developed by the city of Seattle to report non-emergency issues. == History == The City of Seattle launched Find It, Fix It in 2013 for Android and iOS phones to let citizens report potholes, graffiti, and other problems they observe to the city. The app did not support Windows Phone, making it inaccessible to Microsoft employees in the city who used the company's then-supported mobile operating system. In 2015, Mayor Ed Murray led a Find It, Fix It walk with about 100 other people, including police officers, in the University District. Participants were encouraged to use the app to report problems they observed in the neighborhood. Later Find It, Fix It walks have taken place in neighborhoods including Crown Hill, First Hill, Belltown, Wallingford, and Highland Park. In 2020, Find It, Fix It added support for reporting issues with the dockless bicycle sharing systems in the city. Citing the success of Seattle’s app, the nearby city of Kent, Washington, announced that it would create a similar customer service app. == Usage == Users of Find It, Fix It can submit reports about graffiti, potholes, parking violations, broken street signs, and other issues. The app is designed to use a smartphone’s camera and GPS features to make it easier for users to file reports. The Atlantic reported in 2018 that Find It, Fix It was being used by neighborhood groups to report homeless encampments with the intention of having authorities remove them, citing examples of campaigns in Ravenna and Ballard. The executive director of Ballard Alliance, a local chamber of commerce for businesses in the neighborhood, used a private Facebook group to encourage business owners to use the app to report homeless encampments. In response to a poster campaign in the summer of 2019 with the slogan “See a tent? Report a tent”, a representative for the mayor’s office and two Seattle City Council members said that it was inappropriate to encourage use of Find It, Fix It to displace homeless people. As a backlash to these campaigns, people living far from Seattle filed hoax complaints using the app, such as by using photos of tents on display at REI stores. According to the Seattle Times, between January 1, 2020, and November 15, 2021, the city had received over 230,000 service requests, of which 77% were submitted via Find It, Fix It. The largest category of these, numbering over 55,000, concerned illegal dumping. Of complaints categorized as "parking", 3,000 had comments explicitly mentioning issues around homelessness. The ZIP code 98134, covering an industrial area south of Pioneer Square and north of Georgetown, had 5,559 service requests per 1,000 residents, by far the highest in the city.

    Read more →
  • Baum–Welch algorithm

    Baum–Welch algorithm

    In electrical engineering, statistical computing and bioinformatics, the Baum–Welch algorithm is a special case of the expectation–maximization algorithm used to find the unknown parameters of a hidden Markov model (HMM). It makes use of the forward-backward algorithm to compute the statistics for the expectation step. The Baum–Welch algorithm, the primary method for inference in hidden Markov models, is numerically unstable due to its recursive calculation of joint probabilities. As the number of variables grows, these joint probabilities become increasingly small, leading to the forward recursions rapidly approaching values below machine precision. == History == The Baum–Welch algorithm was named after its inventors Leonard E. Baum and Lloyd R. Welch. The algorithm and the Hidden Markov models were first described in a series of articles by Baum and his peers at the IDA Center for Communications Research, Princeton in the late 1960s and early 1970s. One of the first major applications of HMMs was to the field of speech processing. In the 1980s, HMMs were emerging as a useful tool in the analysis of biological systems and information, and in particular genetic information. They have since become an important tool in the probabilistic modeling of genomic sequences. == Description == A hidden Markov model describes the joint probability of a collection of "hidden" and observed discrete random variables. It relies on the assumption that the i-th hidden variable given the (i − 1)-th hidden variable is independent of previous hidden variables, and the current observation variables depend only on the current hidden state. The Baum–Welch algorithm uses the well known EM algorithm to find the maximum likelihood estimate of the parameters of a hidden Markov model given a set of observed feature vectors. Let X t {\displaystyle X_{t}} be a discrete hidden random variable with N {\displaystyle N} possible values (i.e. We assume there are N {\displaystyle N} states in total). We assume the P ( X t ∣ X t − 1 ) {\displaystyle P(X_{t}\mid X_{t-1})} is independent of time t {\displaystyle t} , which leads to the definition of the time-independent stochastic transition matrix A = { a i j } = P ( X t = j ∣ X t − 1 = i ) . {\displaystyle A=\{a_{ij}\}=P(X_{t}=j\mid X_{t-1}=i).} The initial state distribution (i.e. when t = 1 {\displaystyle t=1} ) is given by π i = P ( X 1 = i ) . {\displaystyle \pi _{i}=P(X_{1}=i).} The observation variables Y t {\displaystyle Y_{t}} can take one of K {\displaystyle K} possible values. We also assume the observation given the "hidden" state is time independent. The probability of a certain observation y i {\displaystyle y_{i}} at time t {\displaystyle t} for state X t = j {\displaystyle X_{t}=j} is given by b j ( y i ) = P ( Y t = y i ∣ X t = j ) . {\displaystyle b_{j}(y_{i})=P(Y_{t}=y_{i}\mid X_{t}=j).} Taking into account all the possible values of Y t {\displaystyle Y_{t}} and X t {\displaystyle X_{t}} , we obtain the N × K {\displaystyle N\times K} matrix B = { b j ( y i ) } {\displaystyle B=\{b_{j}(y_{i})\}} where b j {\displaystyle b_{j}} belongs to all the possible states and y i {\displaystyle y_{i}} belongs to all the observations. An observation sequence is given by Y = ( Y 1 = y 1 , Y 2 = y 2 , … , Y T = y T ) {\displaystyle Y=(Y_{1}=y_{1},Y_{2}=y_{2},\ldots ,Y_{T}=y_{T})} . Thus we can describe a hidden Markov chain by θ = ( A , B , π ) {\displaystyle \theta =(A,B,\pi )} . The Baum–Welch algorithm finds a local maximum for θ ∗ = a r g m a x θ ⁡ P ( Y ∣ θ ) {\displaystyle \theta ^{}=\operatorname {arg\,max} _{\theta }P(Y\mid \theta )} (i.e. the HMM parameters θ {\displaystyle \theta } that maximize the probability of the observation). === Algorithm === Set θ = ( A , B , π ) {\displaystyle \theta =(A,B,\pi )} with random initial conditions. They can also be set using prior information about the parameters if it is available; this can speed up the algorithm and also steer it toward the desired local maximum. ==== Forward procedure ==== Let α i ( t ) = P ( Y 1 = y 1 , … , Y t = y t , X t = i ∣ θ ) {\displaystyle \alpha _{i}(t)=P(Y_{1}=y_{1},\ldots ,Y_{t}=y_{t},X_{t}=i\mid \theta )} , the probability of seeing the observations y 1 , y 2 , … , y t {\displaystyle y_{1},y_{2},\ldots ,y_{t}} and being in state i {\displaystyle i} at time t {\displaystyle t} . This is found recursively: α i ( 1 ) = π i b i ( y 1 ) , {\displaystyle \alpha _{i}(1)=\pi _{i}b_{i}(y_{1}),} α i ( t + 1 ) = b i ( y t + 1 ) ∑ j = 1 N α j ( t ) a j i . {\displaystyle \alpha _{i}(t+1)=b_{i}(y_{t+1})\sum _{j=1}^{N}\alpha _{j}(t)a_{ji}.} Since this series converges exponentially to zero, the algorithm will numerically underflow for longer sequences. However, this can be avoided in a slightly modified algorithm by scaling α {\displaystyle \alpha } in the forward and β {\displaystyle \beta } in the backward procedure below. ==== Backward procedure ==== Let β i ( t ) = P ( Y t + 1 = y t + 1 , … , Y T = y T ∣ X t = i , θ ) {\displaystyle \beta _{i}(t)=P(Y_{t+1}=y_{t+1},\ldots ,Y_{T}=y_{T}\mid X_{t}=i,\theta )} that is the probability of the ending partial sequence y t + 1 , … , y T {\displaystyle y_{t+1},\ldots ,y_{T}} given starting state i {\displaystyle i} at time t {\displaystyle t} . We calculate β i ( t ) {\displaystyle \beta _{i}(t)} as, β i ( T ) = 1 , {\displaystyle \beta _{i}(T)=1,} β i ( t ) = ∑ j = 1 N β j ( t + 1 ) a i j b j ( y t + 1 ) . {\displaystyle \beta _{i}(t)=\sum _{j=1}^{N}\beta _{j}(t+1)a_{ij}b_{j}(y_{t+1}).} ==== Update ==== We can now calculate the temporary variables, according to Bayes' theorem: γ i ( t ) = P ( X t = i ∣ Y , θ ) = P ( X t = i , Y ∣ θ ) P ( Y ∣ θ ) = α i ( t ) β i ( t ) ∑ j = 1 N α j ( t ) β j ( t ) , {\displaystyle \gamma _{i}(t)=P(X_{t}=i\mid Y,\theta )={\frac {P(X_{t}=i,Y\mid \theta )}{P(Y\mid \theta )}}={\frac {\alpha _{i}(t)\beta _{i}(t)}{\sum _{j=1}^{N}\alpha _{j}(t)\beta _{j}(t)}},} which is the probability of being in state i {\displaystyle i} at time t {\displaystyle t} given the observed sequence Y {\displaystyle Y} and the parameters θ {\displaystyle \theta } ξ i j ( t ) = P ( X t = i , X t + 1 = j ∣ Y , θ ) = P ( X t = i , X t + 1 = j , Y ∣ θ ) P ( Y ∣ θ ) = α i ( t ) a i j β j ( t + 1 ) b j ( y t + 1 ) ∑ k = 1 N ∑ w = 1 N α k ( t ) a k w β w ( t + 1 ) b w ( y t + 1 ) , {\displaystyle \xi _{ij}(t)=P(X_{t}=i,X_{t+1}=j\mid Y,\theta )={\frac {P(X_{t}=i,X_{t+1}=j,Y\mid \theta )}{P(Y\mid \theta )}}={\frac {\alpha _{i}(t)a_{ij}\beta _{j}(t+1)b_{j}(y_{t+1})}{\sum _{k=1}^{N}\sum _{w=1}^{N}\alpha _{k}(t)a_{kw}\beta _{w}(t+1)b_{w}(y_{t+1})}},} which is the probability of being in state i {\displaystyle i} and j {\displaystyle j} at times t {\displaystyle t} and t + 1 {\displaystyle t+1} respectively given the observed sequence Y {\displaystyle Y} and parameters θ {\displaystyle \theta } . The denominators of γ i ( t ) {\displaystyle \gamma _{i}(t)} and ξ i j ( t ) {\displaystyle \xi _{ij}(t)} are the same ; they represent the probability of making the observation Y {\displaystyle Y} given the parameters θ {\displaystyle \theta } . The parameters of the hidden Markov model θ {\displaystyle \theta } can now be updated: π i ∗ = γ i ( 1 ) , {\displaystyle \pi _{i}^{}=\gamma _{i}(1),} which is the expected frequency spent in state i {\displaystyle i} at time 1 {\displaystyle 1} . a i j ∗ = ∑ t = 1 T − 1 ξ i j ( t ) ∑ t = 1 T − 1 γ i ( t ) , {\displaystyle a_{ij}^{}={\frac {\sum _{t=1}^{T-1}\xi _{ij}(t)}{\sum _{t=1}^{T-1}\gamma _{i}(t)}},} which is the expected number of transitions from state i to state j compared to the expected total number of transitions starting in state i, including from state i to itself. The number of transitions starting in state i is equivalent to the number of times state i is observed in the sequence from t = 1 to t = T − 1. b i ∗ ( v k ) = ∑ t = 1 T 1 y t = v k γ i ( t ) ∑ t = 1 T γ i ( t ) , {\displaystyle b_{i}^{}(v_{k})={\frac {\sum _{t=1}^{T}1_{y_{t}=v_{k}}\gamma _{i}(t)}{\sum _{t=1}^{T}\gamma _{i}(t)}},} where 1 y t = v k = { 1 if y t = v k , 0 otherwise {\displaystyle 1_{y_{t}=v_{k}}={\begin{cases}1&{\text{if }}y_{t}=v_{k},\\0&{\text{otherwise}}\end{cases}}} is an indicator function, and b i ∗ ( v k ) {\displaystyle b_{i}^{}(v_{k})} is the expected number of times the output observations have been equal to v k {\displaystyle v_{k}} while in state i {\displaystyle i} over the expected total number of times in state i {\displaystyle i} . These steps are now repeated iteratively until a desired level of convergence. Note: It is possible to over-fit a particular data set. That is, P ( Y ∣ θ final ) > P ( Y ∣ θ true ) {\displaystyle P(Y\mid \theta _{\text{final}})>P(Y\mid \theta _{\text{true}})} . The algorithm also does not guarantee a global maximum. ==== Multiple sequences ==== The algorithm described thus far assumes a single observed sequence Y = y 1 , … , y T {\displaystyle Y=y_{1},\ldots ,y_{T}} . However, in many situations, there are several sequences observed: Y 1 ,

    Read more →
  • Is an AI Text-to-image Tool Worth It in 2026?

    Is an AI Text-to-image Tool Worth It in 2026?

    Trying to pick the best AI text-to-image tool? An AI text-to-image tool is software that uses machine learning to help you get more done — it scales effortlessly from a single task to thousands. The best picks balance beginner-friendly simplicity with the depth power users need, and they ship updates often. Whether you are a beginner or a pro, the right AI text-to-image tool slots into your workflow and pays for itself fast. Read on for hands-on impressions, pricing tiers, and the standout features that matter.

    Read more →
  • Yejin Choi

    Yejin Choi

    Yejin Choi (Korean: 최예진; born 1977) is the Dieter Schwarz Foundation Professor and Senior Fellow at the Department of Computer Science at Stanford University and the Stanford Institute for Human-Centered Artificial Intelligence (HAI) respectively. Her research considers natural language processing and computer vision. == Early life and education == Choi is from South Korea. She attended Seoul National University. After earning a bachelor's degree in Computer Science, Choi moved to the United States, where she joined Cornell University as a graduate student. There she worked with Claire Cardie on natural language processing. After earning her doctorate, Choi joined Stony Brook University as an Assistant Professor of Computer Science. At Stony Brook University Choi developed a statistical technique to identify fake hotel reviews. == Research and career == In 2018 Choi joined the Allen Institute for AI. Her research looks to endow computers with a statistical understanding of written language. She became interested in neural networks and their application in artificial intelligence. She started to assemble a knowledge base that became known as the atlas of machine commonsense (ATOMIC). By the time she had finished the creation of ATOMIC, the language model generative Pre-trained Transformer 2 (GPT-2) had been released. ATOMIC does not make use of linguistic rules, but combines the representations of different languages within a neural network. In 2020, Choi was endowed with the Brett Helsel Professorship, which she held until she became Chair of Computer Science in 2023. She has since made use of Commonsense Transformers (COMET) with Good old fashioned artificial intelligence (GOFAI). The approach combines symbolic reasoning and neural networks. She has developed computational models that can detect biases in language that work against people from underrepresented groups. For example, one study demonstrated that female film characters are portrayed as less powerful than their male counterparts. In 2023, Choi became The Wissner-Slivka Chair of Computer Science. Choi is also a scientific advisor to French research group Kyutai which is being funded by Xavier Niel, Rodolphe Saadé, Eric Schmidt, and others. In 2025, Stanford HAI announced the appointment of Choi as senior fellow and the Dieter Schwarz Foundation HAI Professor and Professor of Computer Science at Stanford University. == Awards and honours == 2013 International Conference on Computer Vision Marr Prize 2016 Institute of Electrical and Electronics Engineers AI One to Watch 2017 Facebook ParlAI Research Award 2018 Anita Borg Early Career Award 2020 Association for the Advancement of Artificial Intelligence Outstanding Paper Award 2021 Conference on Neural Information Processing Systems Outstanding Paper Award 2021 Association for Computational Linguistics Test-of-time Paper Award 2021 Conference on Computer Vision and Pattern Recognition Longuet-Higgins Prize 2022 North American Chapter of the Association for Computational Linguistics Best Paper Award 2022 International Conference on Machine Learning Outstanding Paper Award 2022 MacArthur Fellowship 2023 Association for Computational Linguistics Best Paper Award 2023 TIME100 Archived 2024-12-27 at the Wayback Machine AI 2023 2023 Empirical Methods in Natural Language Processing Outstanding Paper Award 2025 Association for Computational Linguistics Outstanding Paper Award 2025 Association for Computational Linguistics Best Demo Paper Award 2025 TIME100 AI 2025 == Select publications == Ott, Myle; Choi, Yejin; Cardie, Claire; Hancock, Jeffrey T. (2011). "Finding Deceptive Opinion Spam by Any Stretch of the Imagination". Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies. Portland, Oregon, USA: Association for Computational Linguistics: 309–319. arXiv:1107.4557. Bibcode:2011arXiv1107.4557O. ISBN 9781932432879. S2CID 2510724. Kulkarni, Girish; Premraj, Visruth; Ordonez, Vicente; Dhar, Sagnik; Li, Siming; Choi, Yejin; Berg, Alexander C.; Berg, Tamara L. (2013). "BabyTalk: Understanding and Generating Simple Image Descriptions". IEEE Transactions on Pattern Analysis and Machine Intelligence. 35 (12): 2891–2903. Bibcode:2013ITPAM..35.2891K. CiteSeerX 10.1.1.225.5228. doi:10.1109/TPAMI.2012.162. ISSN 1939-3539. PMID 22848128. Choi, Yejin; Cardie, Claire; Riloff, Ellen; Patwardhan, Siddharth (2005). "Identifying sources of opinions with conditional random fields and extraction patterns". Proceedings of the conference on Human Language Technology and Empirical Methods in Natural Language Processing - HLT '05. Morristown, NJ, USA: Association for Computational Linguistics. pp. 355–362. doi:10.3115/1220575.1220620.

    Read more →
  • Desktop video

    Desktop video

    Desktop video refers to a phenomenon lasting from the mid-1980s to the early 1990s when the graphics capabilities of personal computers such as the Amiga, Macintosh II, and specially-upgraded IBM PC compatibles had advanced to the point where individuals and local broadcasters could use them for analog non-linear editing and vision mixing in video production. Despite the use of computers, desktop video should not be confused with digital video since the video data remained analog, and it uses items like a VCR and a camcorder to record the video. Full-screen, full-motion video's vast storage requirements meant that the promise of digital encoding would not be realized on desktop computers for at least another decade. == Description == There were multiple models of genlock cards available to synchronize the content; the Newtek Video Toaster was commonly used in Amiga in countries that used NTSC (PAL-M in Brazil), while PCs had Truevision and Matrox Illuminator cards and Mac systems had the SuperMac Video Spigot and Radius VideoVision cards. Apple later introduced the Macintosh Quadra 840AV and Centris 660AV systems to specifically address this market. Desktop video was a parallel development to desktop publishing and enabled many small production houses and local TV stations to produce their own original content for the first time. Along with the advent of public-access cable channels, desktop video meant that television advertising became affordable for local businesses such as retailers, restaurants, real estate agents, contractors and auto dealers. As with the phrase desktop publishing, use of the term died out as the technologies to which it referred become the norm for any kind of video production.

    Read more →
  • Nondeterministic finite automaton

    Nondeterministic finite automaton

    In automata theory, a finite-state machine is called a deterministic finite automaton (DFA), if each of its transitions is uniquely determined by its source state and input symbol, and reading an input symbol is required for each state transition. A nondeterministic finite automaton (NFA), or nondeterministic finite-state machine, does not need to obey these restrictions. In particular, every DFA is also an NFA. Sometimes the term NFA is used in a narrower sense, referring to an NFA that is not a DFA, but not in this article. Using the subset construction algorithm, each NFA can be translated to an equivalent DFA; i.e., a DFA recognizing the same formal language. Like DFAs, NFAs only recognize regular languages. NFAs were introduced in 1959 by Michael O. Rabin and Dana Scott, who also showed their equivalence to DFAs. NFAs are used in the implementation of regular expressions: Thompson's construction is an algorithm for compiling a regular expression to an NFA that can efficiently perform pattern matching on strings. Conversely, Kleene's algorithm can be used to convert an NFA into a regular expression (whose size is generally exponential in the input automaton). NFAs have been generalized in multiple ways, e.g., nondeterministic finite automata with ε-moves, finite-state transducers, pushdown automata, alternating automata, ω-automata, and probabilistic automata. Besides the DFAs, other known special cases of NFAs are unambiguous finite automata (UFA) and self-verifying finite automata (SVFA). == Informal introduction == There are at least two equivalent ways to describe the behavior of an NFA. The first way makes use of the nondeterminism in the name of an NFA. For each input symbol, the NFA transitions to a new state until all input symbols have been consumed. In each step, the automaton nondeterministically "chooses" one of the applicable transitions. If there exists at least one "lucky run", i.e. some sequence of choices leading to an accepting state after completely consuming the input, it is accepted. Otherwise, i.e. if no choice sequence at all can consume all the input and lead to an accepting state, the input is rejected. In the second way, the NFA consumes a string of input symbols, one by one. In each step, whenever two or more transitions are applicable, it "clones" itself into appropriately many copies, each one following a different transition. If no transition is applicable, the current copy is in a dead end, and it "dies". If, after consuming the complete input, any of the copies is in an accept state, the input is accepted, else, it is rejected. == Formal definition == For a more elementary introduction of the formal definition, see automata theory. === Automaton === An NFA is represented formally by a 5-tuple, ( Q , Σ , δ , q 0 , F ) {\displaystyle (Q,\Sigma ,\delta ,q_{0},F)} , consisting of a finite set of states Q {\displaystyle Q} , a finite set of input symbols called the alphabet Σ {\displaystyle \Sigma } , a transition function δ {\displaystyle \delta } : Q × Σ → P ( Q ) {\displaystyle Q\times \Sigma \rightarrow {\mathcal {P}}(Q)} , an initial (or start) state q 0 ∈ Q {\displaystyle q_{0}\in Q} , and a set of accepting (or final) states F ⊆ Q {\displaystyle F\subseteq Q} . Here, P ( Q ) {\displaystyle {\mathcal {P}}(Q)} denotes the power set of Q {\displaystyle Q} . === Recognized language === Given an NFA M = ( Q , Σ , δ , q 0 , F ) {\displaystyle M=(Q,\Sigma ,\delta ,q_{0},F)} , its recognized language is denoted by L ( M ) {\displaystyle L(M)} , and is defined as the set of all strings over the alphabet Σ {\displaystyle \Sigma } that are accepted by M {\displaystyle M} . Loosely corresponding to the above informal explanations, there are several equivalent formal definitions of a string w = a 1 a 2 . . . a n {\displaystyle w=a_{1}a_{2}...a_{n}} being accepted by M {\displaystyle M} : w {\displaystyle w} is accepted if a sequence of states, r 0 , r 1 , . . . , r n {\displaystyle r_{0},r_{1},...,r_{n}} , exists in Q {\displaystyle Q} such that: r 0 = q 0 {\displaystyle r_{0}=q_{0}} r i + 1 ∈ δ ( r i , a i + 1 ) {\displaystyle r_{i+1}\in \delta (r_{i},a_{i+1})} , for i = 0 , … , n − 1 {\displaystyle i=0,\ldots ,n-1} r n ∈ F {\displaystyle r_{n}\in F} . In words, the first condition says that the machine starts in the start state q 0 {\displaystyle q_{0}} . The second condition says that given each character of string w {\displaystyle w} , the machine will transition from state to state according to the transition function δ {\displaystyle \delta } . The last condition says that the machine accepts w {\displaystyle w} if the last input of w {\displaystyle w} causes the machine to halt in one of the accepting states. In order for w {\displaystyle w} to be accepted by M {\displaystyle M} , it is not required that every state sequence ends in an accepting state, it is sufficient if one does. Otherwise, i.e. if it is impossible at all to get from q 0 {\displaystyle q_{0}} to a state from F {\displaystyle F} by following w {\displaystyle w} , it is said that the automaton rejects the string. The set of strings M {\displaystyle M} accepts is the language recognized by M {\displaystyle M} and this language is denoted by L ( M ) {\displaystyle L(M)} . Alternatively, w {\displaystyle w} is accepted if δ ∗ ( q 0 , w ) ∩ F ≠ ∅ {\displaystyle \delta ^{}(q_{0},w)\cap F\not =\emptyset } , where δ ∗ : Q × Σ ∗ → P ( Q ) {\displaystyle \delta ^{}:Q\times \Sigma ^{}\rightarrow {\mathcal {P}}(Q)} is defined recursively by: δ ∗ ( r , ε ) = { r } {\displaystyle \delta ^{}(r,\varepsilon )=\{r\}} where ε {\displaystyle \varepsilon } is the empty string, and δ ∗ ( r , x a ) = ⋃ r ′ ∈ δ ∗ ( r , x ) δ ( r ′ , a ) {\displaystyle \delta ^{}(r,xa)=\bigcup _{r'\in \delta ^{}(r,x)}\delta (r',a)} for all x ∈ Σ ∗ , a ∈ Σ {\displaystyle x\in \Sigma ^{},a\in \Sigma } . In words, δ ∗ ( r , x ) {\displaystyle \delta ^{}(r,x)} is the set of all states reachable from state r {\displaystyle r} by consuming the string x {\displaystyle x} . The string w {\displaystyle w} is accepted if some accepting state in F {\displaystyle F} can be reached from the start state q 0 {\displaystyle q_{0}} by consuming w {\displaystyle w} . === Initial state === The above automaton definition uses a single initial state, which is not necessary. Sometimes, NFAs are defined with a set of initial states. There is an easy construction that translates an NFA with multiple initial states to an NFA with a single initial state, which provides a convenient notation. == Example == The following automaton M, with a binary alphabet, determines if the input ends with a 1. Let M = ( { p , q } , { 0 , 1 } , δ , p , { q } ) {\displaystyle M=(\{p,q\},\{0,1\},\delta ,p,\{q\})} where the transition function δ {\displaystyle \delta } can be defined by this state transition table (cf. upper left picture): State Input 0 1 p { p } { p , q } q ∅ ∅ {\displaystyle {\begin{array}{|c|cc|}{\bcancel {{}_{\text{State}}\quad {}^{\text{Input}}}}&0&1\\\hline p&\{p\}&\{p,q\}\\q&\emptyset &\emptyset \end{array}}} Since the set δ ( p , 1 ) {\displaystyle \delta (p,1)} contains more than one state, M is nondeterministic. The language of M can be described by the regular language given by the regular expression (0|1)1. All possible state sequences for the input string "1011" are shown in the lower picture. The string is accepted by M since one state sequence satisfies the above definition; it does not matter that other sequences fail to do so. The picture can be interpreted in a couple of ways: In terms of the above "lucky-run" explanation, each path in the picture denotes a sequence of choices of M. In terms of the "cloning" explanation, each vertical column shows all clones of M at a given point in time, multiple arrows emanating from a node indicate cloning, a node without emanating arrows indicating the "death" of a clone. The feasibility to read the same picture in two ways also indicates the equivalence of both above explanations. Considering the first of the above formal definitions, "1011" is accepted since when reading it M may traverse the state sequence ⟨ r 0 , r 1 , r 2 , r 3 , r 4 ⟩ = ⟨ p , p , p , p , q ⟩ {\displaystyle \langle r_{0},r_{1},r_{2},r_{3},r_{4}\rangle =\langle p,p,p,p,q\rangle } , which satisfies conditions 1 to 3. Concerning the second formal definition, bottom-up computation shows that δ ∗ ( p , ε ) = { p } {\displaystyle \delta ^{}(p,\varepsilon )=\{p\}} , hence δ ∗ ( p , 1 ) = δ ( p , 1 ) = { p , q } {\displaystyle \delta ^{}(p,1)=\delta (p,1)=\{p,q\}} , hence δ ∗ ( p , 10 ) = δ ( p , 0 ) ∪ δ ( q , 0 ) = { p } ∪ { } {\displaystyle \delta ^{}(p,10)=\delta (p,0)\cup \delta (q,0)=\{p\}\cup \{\}} , hence δ ∗ ( p , 101 ) = δ ( p , 1 ) = { p , q } {\displaystyle \delta ^{}(p,101)=\delta (p,1)=\{p,q\}} , and hence δ ∗ ( p , 1011 ) = δ ( p , 1 ) ∪ δ ( q , 1 ) = { p , q } ∪ { } {\displaystyle \delta ^{}(p,1011)=\delta (p,1)\cup \delta (q,1)=\{p,q\}\cup \{\}} ; since that set is

    Read more →
  • Is an AI Humanizer Worth It in 2026?

    Is an AI Humanizer Worth It in 2026?

    Shopping for the best AI humanizer? An AI humanizer is software that uses machine learning to help you get more done — it keeps getting smarter as the underlying models improve. Pricing, accuracy, and the size of the model behind the tool are the three factors that most affect daily usefulness. Whether you are a beginner or a pro, the right AI humanizer slots into your workflow and pays for itself fast. We tested the leading options and ranked them by quality, value, and ease of use.

    Read more →
  • Top 10 AI Chatbots Compared (2026)

    Top 10 AI Chatbots Compared (2026)

    Shopping for the best AI chatbot? An AI chatbot is software that uses machine learning to help you get more done — it keeps getting smarter as the underlying models improve. Pricing, accuracy, and the size of the model behind the tool are the three factors that most affect daily usefulness. Whether you are a beginner or a pro, the right AI chatbot slots into your workflow and pays for itself fast. Below we compare features, pricing, and real output so you can choose with confidence.

    Read more →
  • Pandemonium architecture

    Pandemonium architecture

    Pandemonium architecture is a theory in cognitive science that describes how visual images are processed by the brain. It has applications in artificial intelligence and pattern recognition. The theory was introduced by the artificial intelligence pioneer Oliver Selfridge in his 1959 paper "Pandemonium - A Paradigm for Learning". It describes the process of object recognition as the exchange of signals within a hierarchical system of detection and association, the elements of which Selfridge metaphorically termed "demons". This model is now recognized as the basis of visual perception in cognitive science. Pandemonium architecture arose in response to the inability of template matching theories to offer a biologically plausible explanation of the image constancy phenomenon. Contemporary researchers praise this architecture for its elegancy and creativity; that the idea of having multiple independent systems (e.g., feature detectors) working in parallel to address the image constancy phenomena of pattern recognition is powerful yet simple. The basic idea of the pandemonium architecture is that a pattern is first perceived in its parts before the "whole". Pandemonium architecture was one of the first computational models in pattern recognition. Although not perfect, the pandemonium architecture influenced the development of modern connectionist, artificial intelligence, and word recognition models. == History == Most research in perception has been focused on the visual system, investigating the mechanisms of how we see and understand objects. A critical function of our visual system is its ability to recognize patterns, but the mechanism by which this is achieved is unclear. The earliest theory that attempted to explain how we recognize patterns is the template matching model. According to this model, we compare all external stimuli against an internal mental representation. If there is "sufficient" overlap between the perceived stimulus and the internal representation, we will "recognize" the stimulus. Although some machines follow a template matching model (e.g., bank machines verifying signatures and accounting numbers), the theory is critically flawed in explaining the phenomena of image constancy: we can easily recognize a stimulus regardless of the changes in its form of presentation (e.g., T and T are both easily recognized as the letter T). It is highly unlikely that we have a stored template for all of the variations of every single pattern. As a result of the biological plausibility criticism of the template matching model, feature detection models began to rise. In a feature detection model, the image is first perceived in its basic individual elements before it is recognized as a whole object. For example, when we are presented with the letter A, we would first see a short horizontal line and two slanted long diagonal lines. Then we would combine the features to complete the perception of A. Each unique pattern consists of different combination of features, which means those that are formed with the same features will generate the same recognition. That is, regardless of how we rotate the letter A, is still perceived as the letter A. It is easy for this sort of architecture to account for the image constancy phenomena because you only need to "match" at the basic featural level, which is presumed to be limited and finite, thus biologically plausible. The best known feature detection model is called the pandemonium architecture. == Pandemonium architecture == The pandemonium architecture was originally developed by Oliver Selfridge in the late 1950s. The architecture is composed of different groups of "demons" working independently to process the visual stimulus. Each group of demons is assigned to a specific stage in recognition, and within each group, the demons work in parallel. There are four major groups of demons in the original architecture. The concept of feature demons, that there are specific neurons dedicated to perform specialized processing is supported by research in neuroscience. Hubel and Wiesel found there were specific cells in a cat's brain that responded to specific lengths and orientations of a line. Similar findings were discovered in frogs, octopuses and a variety of other animals. Octopuses were discovered to be only sensitive to verticality of lines, whereas frogs demonstrated a wider range of sensitivity. These animal experiments demonstrate that feature detectors seem to be a very primitive development. That is, it did not result from the higher cognitive development of humans. Not surprisingly, there is also evidence that the human brain possesses these elementary feature detectors as well. Moreover, this architecture is capable of learning, similar to a back-propagation styled neural network. The weight between the cognitive and feature demons can be adjusted in proportion to the difference between the correct pattern and the activation from the cognitive demons. To continue with our previous example, when we first learned the letter R, we know is composed of a curved, long straight, and a short angled line. Thus when we perceive those features, we perceive R. However, the letter P consists of very similar features, so during the beginning stages of learning, it is likely for this architecture to mistakenly identify R as P. But through constant exposure of confirming R's features to be identified as R, the weights of R's features to P are adjusted so the P response becomes inhibited (e.g., learning to inhibit the P response when a short angled line is detected). In principle, a pandemonium architecture can recognize any pattern. As mentioned earlier, this architecture makes error predictions based on the amount of overlapping features. Such as, the most likely error for R should be P. Thus, in order to show this architecture represents the human pattern recognition system we must put these predictions into test. Researchers have constructed scenarios where various letters are presented in situations that make them difficult to identify; then types of errors were observed, which was used to generate confusion matrices: where all of the errors for each letter are recorded. Generally, the results from these experiments matched the error predictions from the pandemonium architecture. Also as a result of these experiments, some researchers have proposed models that attempted to list all of the basic features in the Roman alphabet. == Criticism == A major criticism of the pandemonium architecture is that it adopts a completely bottom-up processing: recognition is entirely driven by the physical characteristics of the targeted stimulus. This means that it is unable to account for any top-down processing effects, such as context effects (e.g., pareidolia), where contextual cues can facilitate (e.g., word superiority effect: it is relatively easier to identify a letter when it is part of a word than in isolation) processing. However, this is not a fatal criticism to the overall architecture, because is relatively easy to add a group of contextual demons to work along with the cognitive demons to account for these context effects. Although the pandemonium architecture is built on the fact that it can account for the image constancy phenomena, some researchers have argued otherwise; and pointed out that the pandemonium architecture might share the same flaws from the template matching models. For example, the letter H is composed of 2 long vertical lines and a short horizontal line; but if we rotate the H 90 degrees in either direction, it is now composed of 2 long horizontal lines and a short vertical line. In order to recognize the rotated H as H, we would need a rotated H cognitive demon. Thus we might end up with a system that requires a large number of cognitive demons in order to produce accurate recognition, which would lead to the same biological plausibility criticism of the template matching models. However, it is rather difficult to judge the validity of this criticism because the pandemonium architecture does not specify how and what features are extracted from incoming sensory information, it simply outlines the possible stages of pattern recognition. But of course that raises its own questions, to which it is almost impossible to criticize such a model if it does not include specific parameters. Also, the theory appears to be rather incomplete without defining how and what features are extracted, which proves to be especially problematic with complex patterns (e.g., extracting the weight and features of a dog). Some researchers have also pointed out that the evidence supporting the pandemonium architecture has been very narrow in its methodology. Majority of the research that supports this architecture has often referred to its ability to recognize simple schematic drawings that are selected from a small finite set (e.g., letters in the Roman alphabet). Evidence from these types of exper

    Read more →
  • Sparse dictionary learning

    Sparse dictionary learning

    Sparse dictionary learning (also known as sparse coding or SDL) is a representation learning method which aims to find a sparse representation of the input data in the form of a linear combination of basic elements as well as those basic elements themselves. These elements are called atoms, and they compose a dictionary. Atoms in the dictionary are not required to be orthogonal, and they may be an over-complete spanning set. This problem setup also allows the dimensionality of the signals being represented to be higher than any one of the signals being observed. These two properties lead to having seemingly redundant atoms that allow multiple representations of the same signal, but also provide an improvement in sparsity and flexibility of the representation. One of the most important applications of sparse dictionary learning is in the field of compressed sensing or signal recovery. In compressed sensing, a high-dimensional signal can be recovered with only a few linear measurements, provided that the signal is sparse or near-sparse. Since not all signals satisfy this condition, it is crucial to find a sparse representation of that signal such as the wavelet transform or the directional gradient of a rasterized matrix. Once a matrix or a high-dimensional vector is transferred to a sparse space, different recovery algorithms like basis pursuit, CoSaMP, or fast non-iterative algorithms can be used to recover the signal. One of the key principles of dictionary learning is that the dictionary has to be inferred from the input data. The emergence of sparse dictionary learning methods was stimulated by the fact that in signal processing, one typically wants to represent the input data using a minimal amount of components. Before this approach, the general practice was to use predefined dictionaries such as Fourier or wavelet transforms. However, in certain cases, a dictionary that is trained to fit the input data can significantly improve the sparsity, which has applications in data decomposition, compression, and analysis, and has been used in the fields of image denoising and classification, and video and audio processing. Sparsity and overcomplete dictionaries have immense applications in image compression, image fusion, and inpainting. == Problem statement == Given the input dataset X = [ x 1 , . . . , x K ] , x i ∈ R d {\displaystyle X=[x_{1},...,x_{K}],x_{i}\in \mathbb {R} ^{d}} we wish to find a dictionary D ∈ R d × n : D = [ d 1 , . . . , d n ] {\displaystyle \mathbf {D} \in \mathbb {R} ^{d\times n}:D=[d_{1},...,d_{n}]} and a representation R = [ r 1 , . . . , r K ] , r i ∈ R n {\displaystyle R=[r_{1},...,r_{K}],r_{i}\in \mathbb {R} ^{n}} such that both ‖ X − D R ‖ F 2 {\displaystyle \|X-\mathbf {D} R\|_{F}^{2}} is minimized and the representations r i {\displaystyle r_{i}} are sparse enough. This can be formulated as the following optimization problem: argmin D ∈ C , r i ∈ R n ∑ i = 1 K ‖ x i − D r i ‖ 2 2 + λ ‖ r i ‖ 0 {\displaystyle {\underset {\mathbf {D} \in {\mathcal {C}},r_{i}\in \mathbb {R} ^{n}}{\text{argmin}}}\sum _{i=1}^{K}\|x_{i}-\mathbf {D} r_{i}\|_{2}^{2}+\lambda \|r_{i}\|_{0}} , where C ≡ { D ∈ R d × n : ‖ d i ‖ 2 ≤ 1 ∀ i = 1 , . . . , n } {\displaystyle {\mathcal {C}}\equiv \{\mathbf {D} \in \mathbb {R} ^{d\times n}:\|d_{i}\|_{2}\leq 1\,\,\forall i=1,...,n\}} , λ > 0 {\displaystyle \lambda >0} C {\displaystyle {\mathcal {C}}} is required to constrain D {\displaystyle \mathbf {D} } so that its atoms would not reach arbitrarily high values allowing for arbitrarily low (but non-zero) values of r i {\displaystyle r_{i}} . λ {\displaystyle \lambda } controls the trade off between the sparsity and the minimization error. The minimization problem above is not convex because of the ℓ0-"norm" and solving this problem is NP-hard. In some cases L1-norm is known to ensure sparsity and so the above becomes a convex optimization problem with respect to each of the variables D {\displaystyle \mathbf {D} } and R {\displaystyle \mathbf {R} } when the other one is fixed, but it is not jointly convex in ( D , R ) {\displaystyle (\mathbf {D} ,\mathbf {R} )} . === Properties of the dictionary === The dictionary D {\displaystyle \mathbf {D} } defined above can be "undercomplete" if n < d {\displaystyle n d {\displaystyle n>d} with the latter being a typical assumption for a sparse dictionary learning problem. The case of a complete dictionary does not provide any improvement from a representational point of view and thus isn't considered. Undercomplete dictionaries represent the setup in which the actual input data lies in a lower-dimensional space. This case is strongly related to dimensionality reduction and techniques like principal component analysis which require atoms d 1 , . . . , d n {\displaystyle d_{1},...,d_{n}} to be orthogonal. The choice of these subspaces is crucial for efficient dimensionality reduction, but it is not trivial. And dimensionality reduction based on dictionary representation can be extended to address specific tasks such as data analysis or classification. However, their main downside is limiting the choice of atoms. Overcomplete dictionaries, however, do not require the atoms to be orthogonal (they will never have a basis anyway) thus allowing for more flexible dictionaries and richer data representations. An overcomplete dictionary which allows for sparse representation of signal can be a famous transform matrix (wavelets transform, fourier transform) or it can be formulated so that its elements are changed in such a way that it sparsely represents the given signal in a best way. Learned dictionaries are capable of giving sparser solutions as compared to predefined transform matrices. == Algorithms == As the optimization problem described above can be solved as a convex problem with respect to either dictionary or sparse coding while the other one of the two is fixed, most of the algorithms are based on the idea of iteratively updating one and then the other. The problem of finding an optimal sparse coding R {\displaystyle R} with a given dictionary D {\displaystyle \mathbf {D} } is known as sparse approximation (or sometimes just sparse coding problem). A number of algorithms have been developed to solve it (such as matching pursuit and LASSO) and are incorporated in the algorithms described below. === Method of optimal directions (MOD) === The method of optimal directions (or MOD) was one of the first methods introduced to tackle the sparse dictionary learning problem. The core idea of it is to solve the minimization problem subject to the limited number of non-zero components of the representation vector: min D , R { ‖ X − D R ‖ F 2 } s.t. ∀ i ‖ r i ‖ 0 ≤ T {\displaystyle \min _{\mathbf {D} ,R}\{\|X-\mathbf {D} R\|_{F}^{2}\}\,\,{\text{s.t.}}\,\,\forall i\,\,\|r_{i}\|_{0}\leq T} Here, F {\displaystyle F} denotes the Frobenius norm. MOD alternates between getting the sparse coding using a method such as matching pursuit and updating the dictionary by computing the analytical solution of the problem given by D = X R + {\displaystyle \mathbf {D} =XR^{+}} where R + {\displaystyle R^{+}} is a Moore-Penrose pseudoinverse. After this update D {\displaystyle \mathbf {D} } is renormalized to fit the constraints and the new sparse coding is obtained again. The process is repeated until convergence (or until a sufficiently small residue). MOD has proved to be a very efficient method for low-dimensional input data X {\displaystyle X} requiring just a few iterations to converge. However, due to the high complexity of the matrix-inversion operation, computing the pseudoinverse in high-dimensional cases is in many cases intractable. This shortcoming has inspired the development of other dictionary learning methods. === K-SVD === K-SVD is an algorithm that performs SVD at its core to update the atoms of the dictionary one by one and basically is a generalization of K-means. It enforces that each element of the input data x i {\displaystyle x_{i}} is encoded by a linear combination of not more than T 0 {\displaystyle T_{0}} elements in a way identical to the MOD approach: min D , R { ‖ X − D R ‖ F 2 } s.t. ∀ i ‖ r i ‖ 0 ≤ T 0 {\displaystyle \min _{\mathbf {D} ,R}\{\|X-\mathbf {D} R\|_{F}^{2}\}\,\,{\text{s.t.}}\,\,\forall i\,\,\|r_{i}\|_{0}\leq T_{0}} This algorithm's essence is to first fix the dictionary, find the best possible R {\displaystyle R} under the above constraint (using Orthogonal Matching Pursuit) and then iteratively update the atoms of dictionary D {\displaystyle \mathbf {D} } in the following manner: ‖ X − D R ‖ F 2 = | X − ∑ i = 1 K d i x T i | F 2 = ‖ E k − d k x T k ‖ F 2 {\displaystyle \|X-\mathbf {D} R\|_{F}^{2}=\left|X-\sum _{i=1}^{K}d_{i}x_{T}^{i}\right|_{F}^{2}=\|E_{k}-d_{k}x_{T}^{k}\|_{F}^{2}} The next steps of the algorithm include rank-1 approximation of the residual matrix E k {\displaystyle E_{k}} , updating d k {\displaystyle d_{k}} and enforcing the s

    Read more →
  • Top 10 AI Headshot Generators Compared (2026)

    Top 10 AI Headshot Generators Compared (2026)

    Trying to pick the best AI headshot generator? An AI headshot generator is software that uses machine learning to help you get more done — it scales effortlessly from a single task to thousands. The best picks balance beginner-friendly simplicity with the depth power users need, and they ship updates often. Whether you are a beginner or a pro, the right AI headshot generator slots into your workflow and pays for itself fast. Read on for hands-on impressions, pricing tiers, and the standout features that matter.

    Read more →