A computer, no matter how fast and complex it might be, will at best simulate consciousness and imitate intelligence since - by the way a computer is constructed and used today – the computer is a generator for objects and states in a virtual reality. Virtual reality adds to but does not overlap with the physical reality (corporeality) of our everyday life. Intelligence and consciousness are products of corporeality and are therefore separate from the virtual world computers produce and play in.
The border between these realities is sharp and clear and can not be transcended. As long as a computer is the medium for a simulation, imitation, visualization it will produce objects well within the virtual world. It will never produce intelligence or show consciousness.
But is the computer doomed to be a medium only? No.
Aren't our brains also just media for the play of sensations, thoughts, feelings? No.
The difference lies in the way of operation. Todays computers have separated areas for data and programs. The software is mainly impressed from the outside and only little (if at all) altered by the flow of data coming from an internal or external process.
In the brain data and 'program' are intertwined results of 'hardware' and states. The homoeostasis of the system balances the relations between data-intake, output, flow and – mode of operation. The sensoric and operations- parts of the system are influenced by the data-stream. The whole system is a balance of states, where input and output are often not clearly distinguishable. There is no well-defined separation of software and data.
Most likely the brain does not operate like a computer: you point out correctly to the software/hardware dichitomy which does not exist in the brain (as far as we know today). This does not mean that a computer, properly constructed, cannot have consciousness (if only traces). "Virtuality" as the principle underlying computers, is nothing more than a smokescreen hiding our ignorance.
Our fundamental problem with the brain and its 'higher functions' is its complexity which prevents us from copying it and simulating its actions. Manfred Gierer has shown that this complexity is a matter of principle, not of technical capability: a simple 'thought' of the human brain implies more digital operations than a computer comprising all atoms of the universe and all their quantum transitions could perform during the billions of years of its possible existence.
Sorry, it seems I just don't know what 'virtuality' really is
This describes exactly what is going wrong. No matter how little we understand about consciousness - we will not progress a single step if we try to model it in the wrong system. Let's ask Manfred Gierer: how much words do you need to describe a symphony (or even a simple child-song) so exact that everybody gets the same as if listening to the real thing? Even if you use as many words as there are particles in the universe(etc. etc.), you will not manage. Why? Because projecting music on words just does not fit.
Projecting brain-activity on a binary computer does not work. You will be able to simulate appearances' of brain-activities, you might succeed in mimicking some output, or in modelling some processes. But you will never get the real product of brain-ctivity. A simulated lake might show all the appearances that you intend to reproduce - but only that. The simulated lake will lack wetness.
Again: what would the 'brain simulator' look like? Perhaps it would be a human brain, but one we understand (if that is ever possible)?
As a positivist I am convinced: let us stop talking. Let us do all out brain research. Let us collect more and more facts and mechanisms about those 1,100 g tissue which comprises all the answers.
As Wolf Singer said: there are much fewer mechanisms in the brain than phenomena. Doesn't this make hope?