AI Articles

FGPAs will replace GPUs in some AI applications (presumably where the body of knowledge is fixed, and won't need to be retrained). They last longer and consume less power:

h t t p s : // bdtechtalks.com/2020/11/09/fpga-vs-gpu-deep-learning/


Many learning models work in tests but not in the real world due to under-specification:

h t t p s : // www .technologyreview.com/2020/11/18/1012234/training-machine-learning-broken-real-world-heath-nlp-computer-vision/


AI system beats supercomputer at fluid dynamics simulation (faster, less power):

h t t p s : // spectrum.ieee.org/tech-talk/computing/hardware/ai-system-beats-supercomputer-at-key-scientific-simulation

The last one about AI replacing CFD is pretty impressive.

A lot of supercomputer purchases have been for fluid dynamic type problems like weather forecasting, air flow around aircraft and other vehicles, nuclear warheads exploding (link) etc.

The basis of this type of problem is that there are a large number of cells in 3 dimensions, and the equation for each cell must mathematically balance with its adjacent cells (up. down, left, right, front, back). In the 1980s, some mathematical improvements enabled this balancing process to be done an order of magnitude more quickly (I can demonstrate two speedups that I worked out for myself on a small 3x3 two dimensional grid if anyone is interested - basically, solving it as a system of linear equations or using linear programming - both of which handily beat the old method of iterating the grid calculations until they balance).

If these systems can be resolved more quickly by AI, this implies to me that they have what chaos theorists call "emergent patterns" which the ML (machine learning) is able to approximate.

An obvious question then arises: does chess have an emergent pattern?

The fact that NNs that do good evaluation approximations are very big, and not highly accurate (they still require game trees to be built, and even then can't always get a draw) implies that if chess has an emergent pattern, it may be a very large one.

Alternatively, there might be a small emergent pattern, but it's not as easy to find as a fluid dynamics emergent pattern.

Being an optimist, I favour the second option: the individual expressions in a fluid dynamics model are going to be simpler than the expressions in a chess position, and hence the emergent pattern easier to find.

Semantic scholar - link.

It's a search engine for scholarly articles. Where you see TLDR next to a found article, the AI has read the article and summarised it in a single sentence for you!

TheSelfImprover wrote:

A lot of supercomputer purchases have been for fluid dynamic type problems like weather forecasting, air flow around aircraft and other vehicles, nuclear warheads exploding (link) etc.

The basis of this type of problem is that there are a large number of cells in 3 dimensions, and the equation for each cell must mathematically balance with its adjacent cells (up. down, left, right, front, back). In the 1980s, some mathematical improvements enabled this balancing process to be done an order of magnitude more quickly (I can demonstrate two speedups that I worked out for myself on a small 3x3 two dimensional grid if anyone is interested - basically, solving it as a system of linear equations or using linear programming - both of which handily beat the old method of iterating the grid calculations until they balance).

If these systems can be resolved more quickly by AI, this implies to me that they have what chaos theorists call "emergent patterns" which the ML (machine learning) is able to approximate.

An obvious question then arises: does chess have an emergent pattern?

The fact that NNs that do good evaluation approximations are very big, and not highly accurate (they still require game trees to be built, and even then can't always get a draw) implies that if chess has an emergent pattern, it may be a very large one.

Alternatively, there might be a small emergent pattern, but it's not as easy to find as a fluid dynamics emergent pattern.

Being an optimist, I favour the second option: the individual expressions in a fluid dynamics model are going to be simpler than the expressions in a chess position, and hence the emergent pattern easier to find.

I think that the small CPU based neural networks that have come out recently in the likes of Stockfish, Komodo, and so on and perform much bettter than traditional expert system based handcrafted type of evaluation do show that emergent patterns exist in chess and are easy to find, compared to a much more complex problem in 3D fluid dynamics.

Theresa May wrote:

I think that the small CPU based neural networks that have come out recently in the likes of Stockfish, Komodo, and so on and perform much bettter than traditional expert system based handcrafted type of evaluation do show that emergent patterns exist in chess and are easy to find, compared to a much more complex problem in 3D fluid dynamics.

Yes - patterns are "easy" to find. What we don't know is the smallest size of pattern (hence evaluation function) that will provide an accurate evaluation of any chess position. I think it will be smaller than most people would expect.

A concise video (2:48) by Richard Feynman explaining why physics is like learning chess rules in bits and pieces. At the end, he explains how learning big new concepts often simplifies what was previously highly complex. IMO it's very possible that chess has such an "emergent pattern":

VERY significant article IMO: mathematicians are working out how to transform non-linear systems into linear ones so that they can be solved on quantum computers. Cracking this problem is the key to "solving everything" IMO:

https://www.quantamagazine.org/new-quantum-algorithms-finally-crack-nonlinear-equations-20210105/

TheSelfImprover wrote:

A concise video (2:48) by Richard Feynman explaining why physics is like learning chess rules in bits and pieces. At the end, he explains how learning big new concepts often simplifies what was previously highly complex. IMO it's very possible that chess has such an "emergent pattern":

Excellent video. Thanks

AI is getting better at writing at a dramatic rate, and will soon not only pass the Turing test, but will be BETTER at writing than domesticated apes - link!

Related to this, more and more AI is becoming a good choice to go to for advice:

* cheaper

* less likely to tell your story at social gatherings

* less likely to have an agency issue or other biases (e.g. political biases)

* organisations tend to deteriorate when the person who made it good leaves. It's possible the same could happen to an application (the new custodians could ruin it), but it might continue to be just as good

Law, financial advice and many other areas will soon be ripe for picking by AI!

Deep NNs now solving difficult differential equations better than other algorithms - link.

Imo if they would play qualitative high chess, that would be a step into right direction.
They do much better then stockfish or other blind brute force engines.
Yes I know it's not brute force but highly selective. But it uses no purpose.
They play only for increasing score.
Without plan.

I am disappointed that chess programmers have given up.
It all began with fritz and ends with fritz methods.

Only that this method of coming deeper into the tree is more clever then this null move preprocessing.
But it's not a different quality of chess.

And in a discussion the programmers answer:
Who cares if the elo increase is there.
Who cares.

Just increase hardware speed and that's it.
No need for different approach.

AlphaFold is now being used to find new approaches to the cure of "neglected diseases" - link.

A good article IMO: I am firmly of the opinion that by the middle of this century (or even sooner) we will have contagious disease under control. During the Covid outbreak, I have become aware of several new technologies that, with more development, could bring down a pandemic disease outbreak.

An article worth reading (the best AI article I've read for a while) - link:

In summary: try to understand NNs by likening them to "kernels" - an older AI model used for classification (e.g. is it a cat or is it a dog?). In a kernel, the data points are mapped into a higher dimensional space, where all cats will be on one side of a linear hyperplane and all dogs will be on the other. When this is mapped back to the original number of dimensions, the hyperplane will no longer be linear, but it will be close to being the simplest possible function for classifying the data.

It has been mathematically proven that a kernel is mathematically equivalent to an NN of infinite size. My opinion, though, is that the equivalence does not hold very well for real world NNs.

Some people want to be 100% sure. So they don't like NN(statistics)

Countries by AI articles - link.

It is surprising that Britain is 2nd ahead of Germany, China and Japan, but it does tend to be the case that we tend to be early on new ideas, but then fall back as the patterns for new business areas become clear (e.g. we were once WELL ahead of most countries on aircraft, cars, computers etc).

Fortunately for us, I see a period of rapid economic change ahead, which will hopefully favour our national strengths.

This could be important - the usage of maths to reduce AI learning time dramatically - but the article requires subscription. If anyone knows what this article is about, please give us a usable link!

https://www.newscientist.com/article/2311525-simple-mathematical-trick-could-slash-ai-development-time-in-half/

Google docs will now be giving its users a lot of help to improve the style of their writing - link.

TheSelfImprover wrote:

This could be important - the usage of maths to reduce AI learning time dramatically - but the article requires subscription. If anyone knows what this article is about, please give us a usable link!

https://www.newscientist.com/article/2311525-simple-mathematical-trick-could-slash-ai-development-time-in-half/

Lots of sites that lead to nowhere or worse when you search the title text (*) but I think I found the source is in your country, Oxford

https://singularityhub.com/2022/03/21/a-simplified-ai-training-scheme-could-cut-development-time-by-half/

and if you want more there is a link to a free preprint given by that article

https://arxiv.org/pdf/2202.08587.pdf

title: "Gradients without Backpropagation
Atılım Gunes¸ Baydin ¨
1 Barak A. Pearlmutter 2 Don Syme 3 Frank Wood 4 Philip Torr 5"


* Simple mathematical trick could slash AI development time in half
Training artificial intelligences to identify faces or digitise text involves thousands or millions of iterations of a two-stage process known as back-propagation, but a new approach could save time, energy and computing power

Watching netflix „the Silicon Valley revolution“

Deepmind's impressive new system GATO - link.

At this point, I don't know anymore about it than what I've read in the article, so I'll brief you on what the article says:

* the leader before GATO was GPT-3, which is an open source (but very well funded) LLM (large language model)

* GATO is intended to be an AGI (artificial general intelligence)

* GATO is a genuine step forward as an AGI, and is massively better for this than anything that has gone before

* however, it still falls well short of human intelligence

TheSelfImprover wrote:

Deepmind's impressive new system GATO - link.

At this point, I don't know anymore about it than what I've read in the article, so I'll brief you on what the article says:

* the leader before GATO was GPT-3, which is an open source (but very well funded) LLM (large language model)

* GATO is intended to be an AGI (artificial general intelligence)

* GATO is a genuine step forward as an AGI, and is massively better for this than anything that has gone before

* however, it still falls well short of human intelligence

Btw - Deepmind is a Google company, and I just want to mention something else Google has done - but this has been done quietly: Google Translate has improved - and not just incrementally. When Mclane posts a link to a German article, and I right click it in Chrome and select "translate", the quality of the translation is now ABSOLUTELY STUNNING. It mostly reads as well-written English.

For me, Google is maintaining its lead in AI.

The quality  of translation is a bit volatile I still find. For instance just now I wanted a translation of Dutch "chargeren". At first, Google thinks it is English and gives a Dutch translation "opladen". But to my mind, it is never used this way in Dutch, maybe by the Flamish. So I change the language to Dutch and then it gives "charge" as first translation. But it is used mainly as the second, "to exaggerate" in Dutch. Nothing to do with batteries and stuff. So still a lot of work to do.

Not that I am a big fan of all the money that is pumped into this, personally. In the end, open source is more 'futureproof' for one thing. GPT 3 is now exclusively licensed by Microsoft and runs only on Azure, as far as I can see there is no open source of any of it but I could be wrong. So much for Elon Musk wanting AI to be democratically developed for everybody. Then he should not have allowed this exclusive licensing nonsense. So much for Elon Musk wanting to save the human race;



For database queries etc. I can see GPT 3 has value. It is a bit like the computer in Star Trek, only not super intelligent yet

Paraphrasing one of the Lord of the Rings movies:

Bilbo Balings, to Gandalf, who has come to visit Bilbo for his 111th Birthday: "We are alright, we have spongecake!".
Gandalf, ducking under the ceiling beam at Bag End to respond: "Just GPT-3, thank you"

Quote :

The GPT-3 model, which came out last year and impressed the entire world with its capabilities, now has an open-source version: GPT-J. It is a language model created by Eleuther AI, a group of researchers who seek to democratize artificial intelligence.

GPT-J, an open-source alternative to GPT-3

Eelco wrote:

Not that I am a big fan of all the money that is pumped into this, personally. In the end, open source is more 'futureproof' for one thing. GPT 3 is now exclusively licensed by Microsoft and runs only on Azure, as far as I can see there is no open source of any of it but I could be wrong.

I think you're being unduly concerned: as you said yourself in the next post, there's already an open-source equivalent. Also, I am confident that GPT-3 is a LONG way from being the last word in AI, or even language processing. We are in a phase in which AI is advancing rapidly, and it would be surprising if GPT-3 was the upper limit - or even close to it.

With my self-improvement hat on: relax, be positive, and adopt a growth mindset!