That’s because the capital-s Singularity as proposed by Verner Vinge is what we’re worried about here. The advent of a technological achievement that forever changes humanity, possibly signalling the end of it.
This does specifically set a barrier, which is a “Point of No Return” when it comes to technology. This is what most people mean when they mean the Singularity. When a program becomes capital-I Intelligent.
Neumann’s original proposal is as limited by mathematics as an LLM itself. The term Singularity has, as is common in the English language, become a larger term to signify a barrier has been crossed. There are other theories beyond the idea that it’s just self-replication gone wild.
You’re trying to reduce what to most people is a moral quandry to pure mathematics. Since my core point is that pure mathematics is not enough to capture the depth and potential of humanity, I’m not going to be swayed by being told it’s just a mathematical function.
I will give you a boost for being interesting, though.
I think the issue here is you’re interpolating a couple different concepts:
Iterated technological self-improvement resulting in exponential growth
Artificial General Intelligence
The threat to humanity from advanced AI
1 is the singularity, 2 and 3 are frequentlyhypothesized consequences of 1. Kinda like extensive use of fossil fuels is one concept, the greenhouse effect is another, and rising sea levels a third. They are related, but distinct, even though one contributes to another.
Combining related concepts under one term dilutes the term and makes it more difficult to effectively communicate. Of course, the moral quandaries are valuable topics of discussion, but the mathematical function is a separate topic, and likewise valuable in and of itself
@agamemnonymous Take it up with Verner, man. The idea’s been popularized in a way that gathers all three, and there’s even theories about a Non-AI Singularity.
That’s because the capital-s Singularity as proposed by Verner Vinge is what we’re worried about here. The advent of a technological achievement that forever changes humanity, possibly signalling the end of it.
This does specifically set a barrier, which is a “Point of No Return” when it comes to technology. This is what most people mean when they mean the Singularity. When a program becomes capital-I Intelligent.
Neumann’s original proposal is as limited by mathematics as an LLM itself. The term Singularity has, as is common in the English language, become a larger term to signify a barrier has been crossed. There are other theories beyond the idea that it’s just self-replication gone wild.
You’re trying to reduce what to most people is a moral quandry to pure mathematics. Since my core point is that pure mathematics is not enough to capture the depth and potential of humanity, I’m not going to be swayed by being told it’s just a mathematical function.
I will give you a boost for being interesting, though.
I think the issue here is you’re interpolating a couple different concepts:
Iterated technological self-improvement resulting in exponential growth
Artificial General Intelligence
The threat to humanity from advanced AI
1 is the singularity, 2 and 3 are frequentlyhypothesized consequences of 1. Kinda like extensive use of fossil fuels is one concept, the greenhouse effect is another, and rising sea levels a third. They are related, but distinct, even though one contributes to another.
Combining related concepts under one term dilutes the term and makes it more difficult to effectively communicate. Of course, the moral quandaries are valuable topics of discussion, but the mathematical function is a separate topic, and likewise valuable in and of itself
@agamemnonymous Take it up with Verner, man. The idea’s been popularized in a way that gathers all three, and there’s even theories about a Non-AI Singularity.
This happens all the time with terms.