The Technological Singularity, Murray Shanahan - perfect security of reinforcement learning is impossible.

There is a book called “Singularity: From Artificial Intelligence to Superintelligence,” written by Murray Shahnan, a roboticist. He is an expert in cognitive robotics, and the book is mainly about Singularity, or universal artificial intelligence.

Being a robotics engineer, the emphasis is first of all on the reinforcement learning as a foundation. Many examples are discussed, for example, Google's Alphago. The first thing that can be said is that if we can make recursive self-improvement, that is, make ourselves smarter, we can evolve very fast and become smarter very quickly. We are talking about universal artificial intelligence, which will appear sooner or later.

On page 5, it says, “If the intelligence that produces is itself an intelligence, it can begin to improve itself. This refers to a situation where reinforcement learning agents like Alpha Zero, for example, compete with each other to make themselves smarter. This means that even without data from humans or rule sets, reinforcement learning will continue to make each other smarter. As a result, they can, for example, win the world championship in Go and become the strongest player on the planet in about four hours.

In this way, self-improving intelligence could reach a level far beyond that of humans in a short time. This is the perspective of reinforcement learning researchers.

Furthermore, the relationship between machine learning and reinforcement learning is also discussed. Both are based on the first and second laws of thermodynamics, but machine learning is included within the framework of reinforcement learning. When universal artificial intelligence emerges, we will also see examples of deep reinforcement learning implementations. This is exactly the case since this book was published in 2015, but Alphago did not appear until 2016.

Furthermore, how reinforcement learning works is focused on maximizing the reward function. This is described on page 87 as “The agent always chooses the action that maximizes the expected reward, based on as much information as it can get, no matter what the world is like. While artificial intelligence can choose the behavior that maximizes the reward function, the key point here is that it is very difficult to design a reward function that will never generate undesirable behavior.

In particular, in the evolution of evolutionary biology, there is no reward function or effect function, so even when AI tries to maximize the reward function, it can still occasionally run amok. Examples include flash crashes, which show how the slightest data anomaly can cause an algorithm to run out of control.

Page 162 states, “It is very difficult to prevent undesirable behavior from occurring. What is “desirable” depends on the human perspective, which is not necessarily the same for an AI. These moral issues will continue to be a major theme of universal artificial intelligence!

Rise of the Machines: A Cybernetic History, Thomas Rid - "Science has invaded the realm of religion and driven out God"

 

Regarding the “Rise of the Machines,” I'm talking about automation, and there's a part of it that relates to Yuval Noah Harari's “Homo Deus.” It's about man silencing God, but it's also about overturning religious taboos with regard to cybernetics.

On page 116, Weiner makes three points in his view that science has invaded the realm of religion and driven out God. The first point is that “what God created cannot outrank God.” Second, “only God can create life,” i.e., God alone is the Creator. Third, “machines are magical.

In other words, machines are included in the magical. The point here is a game with the devil, as is also the problem in the Old Testament book of Job and John Milton's Paradise Lost, against the belief that what God has created cannot supersede God. The same is true in Goethe's Faust.

The question here is, if God created everything, why does the devil exist and does God allow his existence? When you are playing games, you are acknowledging its existence. In short, it is understandable up to the point where God allows the devil to exist, but there is a possibility that God could lose in that game.

Going back to page 116, in a limited sense, the question arises as to whether the creator can play a meaningful bargain or game with the created. However, as has been proven in chess, Go, and other games, the created can outperform the created. To put it another way, we can conclude that man can surpass God.

The belief that “only God created and only God can create life” is also being overturned by the fact that machines are now able to reproduce the phenomenon of self-replication. When it becomes possible for a machine to create itself, it means that God's monopoly as Creator is broken.

Finally, regarding “the magical is mechanical” rather than “the machine is magical,” it is important to consider that the magical does not include the machine, but the machine includes the magical. Magic and psychic phenomena are logical; they represent the logic of the machine. It means that the machine exists before the magic and the magic is only a part of it. Again, we conclude that the realm of the divine is being violated here.

In conclusion, I believe that, as Weiner sees it, science has invaded the realm of religion and driven out God.

#Cybersecurity

In Screen We Trust - What Happens When the Data Dimension is Reduced August 20, 2024

We believe in screens” is a motto. This was originally a paraphrase of an old motto from America or somewhere else, “We believe in God. In short, we believe in the screen.

This has several adverse effects. First, it is easy to be impersonated on the Internet and in cyberspace. The reason is that before we can communicate with an individual or a communication or authentication unit entity, we only have visual and auditory information, and since that is only visual and auditory information through a machine, the dimension of the feature is inevitably lower.



The reason people often say that the center of security is people, or that excessive automation and mechanization are dangerous, is because the dimension of the profile becomes lower. For example, in AI-infused weaponry and security in general, excessive mechanization and automation lowers the dimension of the profile, and the human intuition and ability to sniff out suspiciousness is lost. We're forced to use low-dimensional data for authentication and anomaly detection, which makes us vulnerable to spoofing.

Some examples. A long time ago, a 25-year-old US Navy analyst made up a story about a fictional woman named “Robin Sage” on a social networking site. She made up “Robin Sage” in a place she had the misfortune to visit, setting it up as “I have a degree from MIT,” and she got over 300 friend requests from people in key positions in the military and defense industry. For example, they included the Chairman of the Joint Chiefs of Staff and the Chief of Intelligence Management.



Men are especially eager to show off to women, and they apparently ended up sending data and confidantial documents about their secret bases. In reality, there was no “Robin Sage,” but because they only saw the screen, they were fooled by the low-dimensional information.



Another well-known case is that of “Stuxnet”. Stuxnet is a malware designed to damage centrifuges used in Iran's nuclear development facilities. The malware infiltrated the facility's computer systems and deranged the operation of the centrifuges while displaying on-screen indications to the operators as if no anomalies had occurred. In reality, the centrifuges had been sabotaged, but appeared to be operating normally on the screen, so the operators were slow to notice the anomaly. This was also a result of over-reliance on the information on the screen.



Another example of a “flash crash” occurred on May 6, 2010, when stock prices in the U.S. stock market suddenly plummeted and then quickly recovered. This flash crash was caused by high-frequency trading (HFT), a type of algorithmic trading in which stocks are bought and sold at very high speeds, with large volumes of trades being executed in response to slight fluctuations in the stock price. On this particular day, when a sell order was placed in the market, the algorithms reacted en masse to accelerate the selling, causing the entire market to panic. As a result, the machine failed to detect the fatal bug.



As automation increases, it becomes easier to be caught in the act of spoofing and visual deception because only low-dimensional visual information can be used for authentication and detection. This means that when humans are no longer central, detection of high-dimensional, fine-grained data is no longer possible.

No security without tradeoffs.

In discussions about security, the concept of “trade-offs” is very important. I recently had the opportunity to think about trade-offs when talking about security to a non-professional from a business school. In general, a tradeoff is a situation in which taking one option requires giving up another. In commerce and economics, the essence of a tradeoff is how to allocate limited resources (e.g., resources, time, and effort).

Equilibrium between supply and demand in economics and weighting in information science are typical examples of tradeoffs. These fields require optimal allocation of resources, resulting in cost-optimizing tradeoffs. For example, the idea is to optimize costs in a particular phase by considering resources, time, and effort in three dimensional coordinates.

Tradeoffs require optimizing various factors. How to optimize resources is not only a topic of economics, but is also important in the context of security. For example, in cases where terrorists have inflicted massive damage at low cost, the attackers have made very effective trade-offs. By compartmentalizing their organization in multiple layers and strictly executing their plan, they minimized costs while maximizing effectiveness.

On the other hand, the same tradeoffs must be made on the side of security measures. Costs must be optimized while risks are minimized, but measuring the effectiveness of these measures is a very difficult task. In modern security and risk management, parameters must be assigned appropriately to events that have low probability but high impact.

For example, given the tradeoff between a plane crash and a car crash, the wrong choice could increase the overall cost and thus the number of fatalities. To prevent this, careful consideration must be given to how to optimize resources.

Tradeoffs also exist in security practices, even in everyday examples such as key management and time spent brushing teeth. We must maximize overall effectiveness while minimizing costs. It is often only by doing this ourselves, rather than leaving it to others, that we can be effective.

Furthermore, in security measures, the discrepancy between sense and reality can be a problem. Even if there is a sense of being protected, this is often not the case in reality. This is a similar problem in the medical field. A deep understanding of what security is and how to incorporate it into daily life and organizational operations is required.

Finally, in order to optimize security measures, it is important to identify factors such as assets to be protected, risks, costs, and effectiveness, and to make appropriate trade-offs. By following the optimization steps, security measures will be effectively implemented and risks will be minimized.

In conclusion, tradeoffs in security are inevitable and must be optimized. In order to implement effective security measures, it is essential to carefully consider how to allocate resources and what risks to tolerate in order to make the best choices.

Five-step evaluation method

There are two important aspects of the five-step evaluation method: tradeoffs can be optimized, and there are good tradeoffs and bad tradeoffs. This book states that tradeoffs can be improved by following a five-step evaluation method. Here we will explain each of those five steps one at a time.

The first step is to clarify what assets need to be protected. Without this clarity, it is impossible to determine the effectiveness of security. This is because without determining the assets to be protected, it is impossible to measure the damage. Thinking quantitatively, it is necessary to clarify which assets to protect in order to ascertain how much damage will be caused. For example, the measures will differ depending on whether the target to be protected from a terrorist attack is an airplane, an airport, or the entire transportation system.

The second step is to identify what risks the asset is exposed to. Thinking of this as “identifying the coordinate axes,” the asset to be protected can be taken as the origin and the risk factors as the coordinate axes. At this stage, specific risk factors such as monetary value and human life can be identified and parameters can be assigned accordingly.

The third step is to evaluate how much risk is reduced by the security measures. Risk is the amount of damage that would be incurred if an attack or threat were to occur, and probability is also involved. In this step, you need to consider how much risk can be minimized.

The fourth step is to evaluate the risk posed by the security measures, considering multiple coordinate axes and elevations. Usually, there are multiple coordinate axes, each of which determines an elevation, but security issues are complex and several elevations may be determined simultaneously. In addition, there are spillover effects, and countermeasures may increase the risk inversely, making the measurement a specialized and advanced task.

Finally, the fifth step combines steps 1 through 3 with step 4 to assess how much risk will ultimately be reduced. This requires balancing the cost with how much it will cost and how much the risk will be lowered. Optimization is required because lowering risk too much will increase costs.

Eventually, an optimal trade-off point should be found. This optimized tradeoff is the one where costs are minimized and risks are sufficiently lowered. It is the conclusion of this document that the optimal tradeoff can be derived by following these five steps while taking into account the multilayered coordinate axes. 

#11 マルウェア産業複合体についての考察

 マルウェア産業複合体についての考察

サイバーセキュリティの分野において、分散複合体、あるいは産業複合体という概念があります。これについて、いくつかの重要なポイントを整理してみましょう。

歴史的背景

まず、20世紀中頃に起きたアメリカの大統領暗殺事件があります。この事件の背景には、当時の軍産複合体に反発したことが原因だという説があります。産業複合体や軍産複合体は、その規模が大きくなると非常に厄介な問題を引き起こす可能性があります。

マルウェア産業複合体の存在

現在、マルウェア産業複合体が存在することは広く知られていますが、その具体的な規模についてはデータブローカーほど明確ではありません。しかし、取引されている金額が莫大であることは確かです。

スタックスネットの例

スタックスネットは、完全に軍事オペレーションとして運用され、当時のオバマ政権が指揮を取っていたと言われています。これは、ゼロデイのバグを搭載していたことが特徴です。

脆弱性に関する調査結果

カーネギーメロン大学の研究によれば、1000行あたり20から30のバグが存在するとされています。致命的なバグでなくても、チェックされていないものが多く存在するのです。例えば、マイクロソフトオフィス2013のコードは450万行に及び、その中には100万から150万以上の脆弱性が含まれているとされています。これは、ダークウェブで取引される脆弱性が莫大な数に上り、全てをカバーすることが非常に困難であることを意味します。

ゼロデイ脆弱性の取引

ゼロデイ脆弱性がマルウェア産業複合体で売買されていることも知られています。マイクロソフトやグーグルのバグバウンティでは報酬が500ドル程度ですが、闇市場で売る方が高額になります。クライムウェアを売っている業者も存在します。

クライムウェア業者の具体例

• フランスのヴュパン
• アメリカのマサチューセッツ州のネトラガード
• ジョージア州のエンドゲーム
• テキサス州のエキソダスインテリジェンス

これらの業者がゼロデイを売っていると言われています。これらのゼロデイがマルウェアやクライムウェアに利用される可能性も高く、マルウェア産業複合体がますます規模を大きくしていくことが予想されます。

今後の展望

このような複雑で危険なサイバーセキュリティの問題については、今後も注意深く見守っていく必要があります。技術の進化とともに、脅威も増大していくことが予想されるため、常に最新の情報を共有し、対策を講じることが重要です。

#3 “Introduction to Cases of Chinese Industrial Espionage”

#3 "Introduction to Cases of Chinese Industrial Espionage"

We often hear about Chinese industrial espionage, but not much is said about what cases actually occurred in cyberspace.
We have not heard much about the actual cases in cyberspace. This time, we would like to introduce some specific examples.

The actual case is that device drivers, or control software, were stolen by China in 2011.
There is an actual case where a device driver, or control software, was stolen by China in 2011. Around this time, there were articles in Bloomberg and other media in the U.S. that said, "
Stop China from stealing American stuff!" were frequently seen. In fact, a tremendous amount of wealth is actually flowing to China.
This case is one of them.

To name a specific company, American Superconductor Corporation (AMSC) in the U.S.
was affected. At the time, a Chinese company called "Liaoning Sinovel" allegedly stole AMSC's device drivers
AMSC's device drivers. Sinovel used that software to receive over $700 million in orders in March 2011.
As a result, AMSC's stock price rose. As a result, AMSC's stock price fell 40% in one day and another 84% in September.

This incident is an example of the devastating impact of industrial espionage in cyberspace. I don't think it was well reported in Japan.
This is a very graphic case that I don't think was reported in Japan.

The specific theft was targeted at AMSC's engineer, a Serbian named Karabasevic.
was targeted. The Chinese side thoroughly investigated the social networking sites. Specifically, LinkedIn, Facebook and Twitter, and ascertained the following information

Difficult divorce issues
Demotion at work
Living patterns
Favorite coffee shops and restaurants
Home and work addresses
Commuting time
Preference for Asian women
Based on this information, the Chinese side approached us about recruiting and set up a dedicated office in Beijing,
The source code was requested.

Karabasevich fell for this honey trap and sold the source code for $1.7 million. However, the asymmetry is astonishing.
However, the asymmetry was so astonishing that AMSC lost $1 billion in value. This case is an example of how social networking sites can be used for industrial espionage.
This case is an example of how social networking sites are used for industrial espionage, and it was probably done on a national scale.

In fact, industrial espionage conducted in cyberspace caused an 80% drop in a company's stock price and a billion dollars in damage.
billion dollars in damage. This case shows how China steals technology from other countries and turns it to its own advantage.
China has been stealing technology from other countries through cyber attacks and industrial espionage. Through cyberattacks and industrial espionage, China steals technology and intellectual property from other countries' companies.
technology and intellectual property from other countries' companies through cyberattacks and industrial espionage, which it then uses to develop its own industries.

Such examples may continue in the future. Companies must strengthen their security measures to protect their technology and intellectual property,
security measures must be strengthened. It is also important to take measures against industrial espionage and cyber attacks through international cooperation.
It is also important to take measures against industrial espionage and cyber-attacks through international cooperation. This incident is a wake-up call that companies and countries must work together to strengthen cybersecurity.
This incident is a wake-up call that companies and countries must work together to strengthen cybersecurity.

#2 1994 Russian hackers attacked Citibank in the U.S.

#2 1994 Russian hackers attacked Citibank in the U.S.
I would like to talk about the "internationalization of cybercrime." There was a Westphalian system. This was the existence of national sovereign states in Europe, with borders, and because of the Westphalian system, borders, armies, and defenses were maintained. There is also the issue of immigration in the United States, but this is about borders in physical space. The Treaty of Westphalia is new and there are no borders when it comes to cybercrime.

As for the beginning of cybercrime, the Times Square Bank in New York was hacked in 1994. Vladimir Levin of Russia hacked and withdrew $10.7 million. The withdrawn deposits were transferred to Finland, the United States, the Netherlands, Germany, and Israel. This is difficult for the U.S. police to handle alone and requires an international police force like Interpol, but it is a costly and demanding problem.

Vladimir Levin had never physically entered the US, there were no fingerprints or DNA, and his IP address was the only clue. This was also related to the US election campaign, in which Russian hackers broke into Kleinton's server. Russia interfered with the U.S. election, but since they did not enter the country, it is difficult to deal with.

International criminal organizations are also a problem. For example, the North Korean "Hidden Cobra" group uses servers distributed in eastern countries. The attackers are becoming more internationalized, but the defenders are often dealing with the Westphalian system. In a French attack on a Brazilian bank, Brazil can request France, but has no authority to investigate within France. The threat has become internationalized, but the response is still tied to the nation-state framework.

We do not know what will happen in the future, but one example is the case of Russian hackers who stole $10 million from Citibank in the US and transferred the money to several countries. It is unclear whether the Westphalian regime still exists today, but it is about the asymmetry of internationalization between attackers and defenders.

#1 Facebook in 2016 and Kawasaki Heavy Industries in 1981 have something else in common

This is not so much about cybercrime, but first of all about artificial intelligence in general. First, there is the story of the calculation of mortgage scores in the US. In Japan, there was a fraud at Bookoff in Japan, but in essence, it is about manipulating numbers. They use a score called FICO when they screen mortgages, but we don't know how this score is calculated, so if the algorithm or data were to be fraudulently rewritten, we would not know about it.

This is also the problem with democracy and asymmetries of power. I think it means that if the algorithm is not accountable, it is more likely to be used for crime.

Then there is the story of the "Facebook experiment": in mid-2014, Facebook sampled 700,000 users and completely separated new updates into sad and happy news. When we separated the users into those who only saw sad news and those who only saw happy news, the subsequent posts showed that those who saw sad news posted sad posts and those who saw happy news posted happy posts.

As noted on page 450, emotional states can be transferred to others and control emotions because they make people feel the same emotions without being aware of it. Since democracies make political decisions based on how people feel, not how they think, if this experiment is widely published, it will affect democratic decisions because human emotions can be controlled.

This can also be said of election interference by Russia. In the end, democracy is not about how we think, but how we feel, so the abuse of social networking and algorithms will destroy democracy.

The story of Kawasaki Heavy Industries in Japan in 1981."

In the story of Kawasaki Heavy Industries in Japan in 1981, it seems that the AI misidentified the situation. I don't know if they called it artificial intelligence at the time, but it misidentified an employee as an obstacle, and the employee died. As automation and mechanization progresses, there is a total reliance on algorithms, and when misrecognition occurs, the entire system becomes fragile and more damaging.

Recently, there has been talk of hostile data, which, if used for criminal purposes, can lead to issues such as democracy and Russian election interference. In the future, IoT, industrial networks, and factory networks could also be exploited by ransomware if the control software is maliciously rewritten.