As outlined above, the coding sequence is the mRNA equivalent of the gene, being defined by genetic analysis carried out at the level of the phenotype. The outcome of this analysis constitutes the genotype as the ensemble of defined inherited functions. Such physiological functions are based on the expression of an ensemble of unit functions. The unit function, subject to mutation, is carried by the polypeptide in its nascent2 form (see Fig. 1). The actual function is exerted in general by a quaternary protein complex, which may integrate several identical and/or different proteins, possibly modified chemically, as well as by low Mr co-factors of organic or inorganic chemical nature.

The unit of a coding sequence is the triplet of nucleotides which, according to the genetic code, directs during translation of an mRNA the choice of a given anticodon carried by a given tRNA. Due to the degeneracy of the code, incorporation of an identical amino acid (aa) may be directed by different triplets. Within our argument, an essential feature based on this fact is that, according to the triplet chosen for a given amino acid, a different nucleotide sequence is formed at the level of the mRNA. In consequence, a different secondary structure of the nucleic acid arises which may be “recognised”, for instance by proteins or interfering RNAs interacting with the RNA.

On the other hand, given amino acids and hence triplets are not equivalent within the polypeptide chain. Indeed, the same type of amino acid may assume different “functions” at the level of the secondary protein structure, in terms of hydrogen-bonding or ionic interaction, once the polypeptide chain is folded in the 3D space. This type of function may hence be projected back onto the corresponding genomic sequence. To single out a given triplet, its position within a coding sequence and/or exon should be labelled. One may hence conceive a notation for the position of a given triplet within a coding sequence. A possible and efficient description, compatible with alignment schemes in bioinformatics, is the following: chromosome / genomic domain / maximal open reading frame / exon / triplet position within exon. (One should note, however, that there is as yet no generally agreed and universally employed convention in bioinformatics for describing the position of a triplet in the genome of a species.)

Accordingly, a given triplet (formally or as a physico-chemical entity) can be followed from the genomic DNA to a collection of amino acids within polypeptides. This is an essential feature when handling the triplet and its information content by a mathematical approach.

By definition, structural protein genes contribute to cellular structure and function either directly or via enzymatic activities. They may constitute the building blocks of the nuclear and plasmatic membranes, the endoplasmic reticulum, the nuclear matrix and the cytoskeleton. As enzymes they govern the intermediary metabolism as well as protein, RNA or lipid biosynthesis and degradation. There are the proteins acting as the mechanistic and enzymatic carriers of the system of protein biosynthesis, which do not discriminate among specific types of DNA, pre-mRNA or mRNA. Among the latter are, for example, the RNA polymerases, the non gene-specific splicing factors, the non-specific transport factors as “exportin” or NLS (nuclear localisation signal) (Rodriguez et al. 2004) binding to RNA sequences, the translation initiation and elongation factors, the poly(A)- (Grossi de Sa et al. 1988) and CAP-binding proteins (Furuichi and Shatkin 2000) (see review in Shatkin and Manley 2000).

Regulatory protein genes control gene expression from chromatin activation to transcription and translation; they may function as repressors and activators of transcription, or act at post-transcriptional levels by interaction with pre-mRNA and mRNA. Four sets of such regulatory proteins can be distinguished: (1) the non-histone type chromatin proteins, as the transcription factors (TFs; see Latchman 1990; Martin 1991) as well as the histone- and DNA-modulating factors which control local remodelling of chromatin, allowing or not accessibility of the transcription machinery to DNA (Felsenfeld 1992; Felsenfeld and Groudine 2003) (for a review see, Gasser 2002; Kouzarides 2007); (2) the nuclear pre-mRNA binding proteins which interact, in specific sets, with given types of pre-mRNA, in statu nascendi (Daneholt 2001; Dreyfuss 1986; Dreyfuss et al. 2002; Maundrell and Scherrer 1979) as well as at the level of the nuclear matrix (De Conto et al. 2000; Maundrell et al. 1981; Razin et al. 2004). There are several hundreds of relatively acid proteins bound by hydrophobic bonds, and relatively fewer (some dozens) basic ones binding possibly by ionic interaction (the “histone-type” of pre-mRNP proteins) (Maundrell and Scherrer 1979); (3) the cytoplasmic proteins binding in variable sets the non-translated mRNAs (Civelli et al. 1980; Spohr et al. 1970; Vincent et al. 1977, 1981); these proteins bind in general by hydrophobic interaction, they act positively in guiding the cytodistribution of mRNA (Arcangeletti et al. 2000; De Conto et al. 1999; Maundrell et al. 1979), and negatively as cytoplasmic repressors (Civelli et al. 1980; Maundrell et al. 1979; Vincent et al. 1983); (4) the prosome particles (Martins de Sa et al. 1986; Schmid et al. 1984; De Conto et al. 2000, 1999; Ioudinkova et al. 2005) (review in Scherrer and Bey 1994), a population of protein complexes built of 2 × 14 subunits in variable composition, which bind on one side to chromatin, pre-mRNA and cytoplasmic repressed mRNA, and on the other to the nuclear matrix and the cytoskeleton (cf. Figs. 6, 7). (Most interestingly, these same 20S particles act as the core of the 26S proteasomes, the main catabolic system able, in conjunction with the ubiquitin system, to degrade selectively specific proteins, Coux et al. 1996). Fig. 6

Transcript size and genomic domainsA The size of giant transcripts (up to 50–100,000 nt) corresponds—by order of magnitude—to the genomic domains observable in specific types of chromosomes B, C, or the “Christmas trees” of primary transcripts observable in the EM after spreading of nucleoli (D1, 2) or non-ribosomal chromatin (D3). (For exp. details see, Scherrer and Darnell 1962 and Scherrer et al. 1963, reporting the original observation of “giant” RNA and RNA processing; cf. also Fig. 9 in Scherrer and Marcaud 1968 and Fig. 6 in Spohr et al. 1976). BLampbrush chromosomes of Pleurodeles waltl stained for IIF with anti-prosome monoclonal antibodies (for exp. details see Pal et al. 1988). Lampbrush chromosomes are characteristic of the transcription of the entire genome during the diplotene stage of oogenesis in amphibia and birds. Projecting from the chromosome axis are the chromatin loops corresponding to genomic domains, which carry the “Christmas trees” of DNA in maximal transcription (comparable to those shown in panel D3). Prosomes (insert) are protein particles (built of 2 × 14 subunits in 4 superposed rings of 7) found associated to chromatin and (pre-)mRNP complexes; they constitute also the core of the 26S proteasomes (Scherrer and Bey 1994). Notice their association to the loops (maximal at their basis), and also their shedding from the chromosomes into the nucleoplasm. CPolytene chromosomes of Rynchsciara americana in specific stages of larval development and differentiation (cf. Glover et al. 1982; Lara 1987). Polytene chromosomes represent interphase chromosomes generated by DNA replication without cell division; about 10,000 DNA strands stay associated and form the bands visible in the light microscope due to chromatin hyper-condensation. These physical bands correspond to the meiotic genes in cytogenetics of, e.g. Drosophila, to units of transcription and, in sciaridae, of DNA replication. Notice the development of transcriptional “puffs” at specific stages of differentiation. DTranscription and formation of nucleoli (relation of transcription and nuclear architecture). 1Organised nucleolus with its fibriller centre F where transcription takes place and the granular zone G constituted by already processed ribosomal subunits. 2Hamkelo-Miller spreads of dissociated nucleoli allow to see consecutive ribosomal DNA domains in transcription: the ribosomal transcripts form RNPs, which, eventually, are organised, into the nucleolar dynamic architecture. 3Transcripts of non-ribososomal genomic domains of various sizes

The most important member of this class of RNA is the ribosomal RNA (rRNA), which serves as the scaffold of ribosomal subunits by organising the sequential alignment of ribosomal proteins (Scheer and Hock 1999; Tschochner and Hurt 2003). The rRNA has, in addition, ribozyme functions (Steitz and Moore 2003). The metabolic precursor of the rRNAs found in the small (16S and 18S rRNA, respectively, in prokaryotes and eukaryotes) and large (23S and 28S rRNA) ribosomal subunits is the nascent pre-rRNA (45S in eukaryotes, Scherrer et al. 1963; Spohr et al. 1976). It is of the same nature aligning, in addition to the proteins ending up in the final ribosome, proteins which, in eukaryotic cells, never leave the nucleolus (Scheer and Benavente 1990; Tschochner and Hurt 2003). They have a structural role in ribosome biosynthesis and the nucleolar dynamic architecture; pre-ribosomes form the fibrillar centre of the nucleolus, whereas the final ribosomal subunits constitute its granular zone (F in Fig. 6D).

Among the RNAs intervening in control of gene expression we have to distinguish those which handle many types of (pre-)mRNA without individual selection, in contrast to those which selectively recognise and control, in a sequence-specific manner, individual types of (pre)mRNA. The latter allow strict recognition and control of individual gene expression, whereas the former RNA may discriminate among classes of, but not of individual mRNAs.

Non-discriminating RNA regulators The most straightforward example of such RNA is the tRNA class which select individual triplets in the coding sequence. Availability of specific types of tRNA corresponding to types of (degenerate) triplets, or the many chemically modified tRNA types may influence and coordinate the expression of classes of mRNA.Similar limited regulatory function is exerted by the U-type RNAs involved in splicing (Valadkhan 2005; Will and Luhrmann 2005). The snoRNAs (small nucleolar RNA) guide modifications (including ribose o-methylation) of (pre-)rRNA and, possibly, even mRNA; they are tissue-specific in higher eukaryotes (Dennis and Omer 2005) but are present already in Archaea. A particular feature of snoRNAs in higher eukaryotes is that they are often encoded in introns of pre-mRNA (Filipowicz and Pogacic 2002).

Discriminating RNA regulators: siRNA and miRNA The advent of RNA interference (RNAi) marked the unexpected discovery of sequence-specific mRNA silencing by natural antisense transcripts (Sontheimer 2005; Tang 2005). This type of post-transcriptional regulation may occur also at pre-mRNA level in the nucleus (Matzke and Birchler 2005).The basic mechanism of RNAi is the synthesis, by an RNA-dependant RNA polymerase and an RNA replicase, of double stranded RNA copies of target RNAs, in particular of mRNA. From such RNA double-strands, several hundred bp long, short 21–25nt long fragments are cut out within the RISC RNA–protein complex. Two classes of interfering RNA are reported, the small interfering RNA (siRNA) and the micro RNA (miRNA) which form distinct siRISC and miRISC complexes (for a recent review see, Sontheimer and Carthew 2005). SiRNAs induce destruction of the target mRNA after sequence-specific hybridisation whereas miRNAs silence temporally the target mRNA.

Proteins cover all types of RNA in the cell. In case of mRNA and pre-mRNA, it was shown at an early time by electron microscopy that proteins are aligned all along the RNA molecules (Dubochet et al. 1973), protecting specific sequences from degradation by RNase (see, Goldenberg et al. 1979 and Fig. 4, insert B). By mass, messenger ribonucleoproteins (mRNPs) include 3 times more protein than RNA. One of the roles for such proteins is to protect RNA from degradation by different types of RNases, which are natively active and abundant in the cellular sap; naked RNA is hence rapidly destroyed.

RNP-type proteins bind in a RNA-sequence dependant manner. The poly(A)-binding proteins (PABPs), attached to the 3′-side tail (length: 50–200 A residues) of the mRNA protect about 12–20 A-residues at a time (Baer and Kornberg 1983); larger RNA-binding proteins may cover up to 50 nt, as is the case, for the viral large T antigen of SV-40 and Polyoma virus (Darlix et al. 1984). The RNP-type proteins include amino acid sequence motifs, recognising the sequence motifs in DNA and RNA. The mRNA, hence, includes sequentially such protein-binding oligomotifs. Actually, only in rare cases the oligo-nucleotide sequence is known that binds a given protein. In addition to the PABP one might mention, the Iron Response Element binding protein (IRE-BP), a protein binding a motif in the 5′-side or 3′-side UTR of the mRNA, respectively for Ferritin and Transferrin (Thomson et al. 1999). There is not even a conventional term to name such motifs in RNA and DNA; we hence introduced the term “oligomotif” (Scherrer and Jost 2007). For the recognition motifs in proteins, one might use the term “aa-motifs”, reserving that of oligomotif for DNA and RNA.

Early proteomic studies of 20 years ago allowed people to estimate that there are several hundred (up to 1,000) acidic, non-histone proteins attached to DNA, and as many to pre-mRNA and FDTs (Maundrell and Scherrer 1979). In the cytoplasm of a given cell, there may exist 500–1,000 different proteins in the repressed mRNPs; a specific mRNA binds a specific combination of such proteins (cf. review in Scherrer and Bey 1994). When the ribosomes are split off the mRNA in polyribosomes in vitro, the translated mRNA is found also in mRNP form, which includes in mass 3 times more protein than RNA. The proteins characteristic for the translated stage are of much fewer, a few dozen types, and seem to be ubiquitously bound to different mRNAs, recognising in particular the 5′- and 3′-side UTRs; a translation-specific PABP binds the Poly(A) tail (Edmonds 2002; Grossi de Sa et al. 1988; Shatkin and Manley 2000).

These observations indicate that there must be a “code” governing the interaction of a limited number of NABPs in chromatin and mRNPs which, in general, are specifically DNA- or RNA-binding proteins. Relatively new data have confirmed, however, the old observation that the same protein may bind both, DNA and RNA, as outlined above. This was originally observed for the large T-antigen of SV 40 and polyoma virus (Darlix et al. 1984; Khandjian et al. 1980) and more recently confirmed for a series of DNA-binding MAR proteins (von Kries et al. 1994)), by sequence identified as, already known, pre-mRNA binding proteins (pre-mRNP or HnRNP proteins); this may represent an interesting exception rather than the rule. The existence of different rules for binding of proteins to DNA and RNA must be assumed. Within this discussion we may use the terms of DNP- and RNP-code (Auweter et al. 2006) for the system governing protein–NA interactions. Specific binding may be by sequence-motifs of about 15 nt minimal length for which we have coined above the neologism of “oligomotif”. An oligomotif would thus interact with an aa-motif in a protein; this interaction can occur either directly, or imply a mechanism of “induced fit”. In terms of information processing, in this exchange, the holo-transgenon generating the binding factors would represent the sender, and the oligomotif in DNA and RNA the receiver of these signals, acting according to the rules of chemical thermodynamics.

In addition to the signals encoded in the oligomotifs of the primary RNA sequence, there are post-transcriptional modifications of the transcripts (review in Shatkin and Manley 2000) which may be recognised by the factors in trans. There is internal methylation of mRNA (Perry and Kelley 1975; Perry and Scherrer 1975) as well as “Capping” (review in Banerjee 1980), of the 5′-side of (pre-) mRNA (consisting in the 5′–5′ addition of GTPs including differential methylation), and there is polyadenylation of the 3′-side (Munroe and Jacobson 1990). Poly(A)_50–200 is recognised by the PABPs of several types which are different in case of the nuclear pre-mRNA, the cytoplasmic repressed and, eventually, the translated mRNA (Mangus et al. 2003). The existence of a poly(A) tail and the corresponding PABP is a factor essential for translation (Gorgoni and Gray 2004; Grossi de Sa et al. 1988). During processing, the transcripts may be cleaved and the site of scission recapped and polyadenylated; primary transcripts may extend far beyond the aauaaa polyadenylation site. Indeed, there are enzymatic systems known to add monophosphates and the 5′–5′-triphospates on the 5′-side (Barbosa and Moss 1978; Venkatesan et al. 1980), as well as poly(A) on the 3′-side of the processing products (Shatkin and Manley 2000). By definition, all these processes are post-transcriptional; thus, the enzymes carrying out these modifications are to be accounted for as factors involved in the generation of the trans-program, implemented by the holo-transgenon of a cell.

The transgenons carried by cP-genes are constituted by the normal mechanisms of gene expression and regulation by protein biosynthesis.

The second mechanism - recently discovered - of transient or final repression of specific mRNAs is RNA interference. Si- and miRNAs might block mRNA upon import to the cytoplasm, or during translation when mRNA segments become accessible as pointed out above. The phenomenon of RNA interference is at present most actively investigated and no general conclusions seem possible as yet. Actually, little could be said with any chance of precision, beyond the general considerations outlined above (see “Discriminating RNA regulators: siRNA and miRNA” section).

It is, however, evident that RNA interference represents at the same time a highly gene-specific system of control, liable to recognise precise RNA targets. It is hence at the same time more efficient but also less sophisticated than the regulatory protein factors. Indeed, the latter are capable to integrate controls to a much higher extent. The si- and miRNAs may represent primitive slots operating in an on/off mode. But this system as well has to be managed upstream by protein factors, not only enzymatic system involved in its generation, but also mRNP proteins. Being single as well as double stranded, interfering RNA is a target for any type of (pre-)mRNA binding protein as well.

RNA interference is likely to have evolved prior to RNA-binding proteins, possible already in pre-biotic systems. RNA hybridisation is the most basic process of RNA stabilisation and neutralisation. Later, chemically more sophisticated systems of RNA-protein recognition and mutual stabilisation may have evolved, much before the tRNA based protein-coding revolution happened, opening the gate to life and evolution.

For the purpose of applying information theory to gene expression, we should first discuss the concept of information itself. Our starting point will be the theory of Shannon. In that theory, a sender composes a message from the elements of a code agreed upon with the receiver. The receiver knows the probabilities p_i with which the individual code words i appear, but apart from that, he does not know the content of the message before receiving it. Thus, before receiving the message, his uncertainty about the actual content of the message to be received is given by the entropy 1\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$ I=-\sum_{i} p_{i} \log p_{i} $$\end{document}where we take the binary logarithm (that is, log 2 = 1). 3 The standard convention 0 log 0 = 0 expresses the fact that no information is gained from events that cannot occur. Also, when some p_i = 1 (and consequently all the other p_j = 0), we use 1 log 1 = 0, meaning that when we are already certain about an event, we do not gain information either.

The formula (1) leads to equating the information content of the received message with the uncertainty present before receiving it, and that uncertainty is quantified by the entropy (1). In that sense, the information received is a reduction of uncertainty. Uncertainty, as expressed by the entropy (1), is converted into information. This is similar to a conversion of potential energy into kinetic energy. The reduction of one is the gain of the other, and the quantities are the same, and are measured in the same units. Thus, for our purposes, entropy is equivalent to potential information.

In our applications to molecular biology, we shall be concerned with sequences (of nucleotides or amino acids). For such a sequence, we want to know its composition, that is, we want to know which element (nucleotide or amino acid , resp.) occurs at each position. This is the information we are after. For formalizing this, there exist two alternative approaches, and in this section, we want to discuss those. One approach consists in simply taking the set of all possible sequences under the given circumstances as an ensemble and then quantify how much information is needed to specify a particular sequence within this ensemble. The other approach looks at the individual positions in the sequence in turn and quantifies how much information is needed to specify which nucleotide or amino acid occurs at that particular position. When we do this for each position and take correlations between the various positions into account, we can again quantify the information needed to determine the composition of our sequence. We shall now describe these two approaches in more formal terms.

Suppose that we are given an ensemble of N items of M different types x with relative frequencies or probabilities p(x).4 The information about the size of the ensemble is given by log₂N. Since this is so simple we shall mostly suppress it in the sequel. The ensemble entropy is then given by2\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$ I=-\sum_x p(x) \log p(x) $$\end{document}Without further knowledge, all the relative frequencies p(x) should be assumed equal, according to Jaynes’ principle of maximal ignorance, and3\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$ I = \log M. $$\end{document}

This is the maximal possible value of I, given the number of types. Refinements through additional knowledge then decrease the entropy; examples include

observations of relative frequencies, restriction of the ensemble, or

Here, without further knowledge, all the p_a are equal (=1/|A|) so that6\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$ -\sum_{\rm a} p_{\rm a} \log p_{\rm a} = \log |A|, $$\end{document}and7\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$ I_S=n \log|A|. $$\end{document}

Since there are M: = |A|ⁿ different such sequences, this is the same as the ensemble entropy log M above, cf. (3). Again, refinements through additional knowledge decrease entropy; examples include

unequal distribution of the p_a, in which case (5) becomes smaller than (7), or

The application of information theory to molecular biology has been controversial. To clarify the issue, the following point might be useful. Usually, information theory is applied to messages. A message contains information when before receiving it one does not know the sequence of symbols in the message, that is, once the message is known that previous uncertainty is reduced. Shannon’s information measure quantifies that reduction of uncertainty, that is, the difference in knowledge before and after receiving the message. This suggests that, likewise, a stretch of DNA contains information about polypeptides or phenotypic properties because knowing that DNA sequence allows one to deduce the composition of those polypeptides or those phenotypic properties. Of course, because of the intervention of other factors, the knowledge of the DNA does not lead to complete knowledge of the relevant polypeptides or phenotypes. The point is, however, that knowing the DNA reduces the uncertainty about those polypeptides or phenotypes, and this then leads to a quantification of the information contained in the DNA. The remaining uncertainty then is assigned to other factors, and the corresponding information can then also be quantified.

The point we are emphasizing here in order to avoid misconceptions about genetic information [see e.g. Stegmann (2005) for a recent discussion] is that for quantifying information one needs to specify first about what there is uncertainty. Uncertainty about sequence identity is different from information about types and numbers of polypeptide chains in a cell, and consequently, the information content is different as well. Below, we shall treat those different situations in turn. More precisely, we shall look at sequence, product, and process information. In each case, the information measure will be different.

In information theory, the message from the sender to the receiver has to pass through a channel, and the latter may not faithfully transmit everything emitted by the sender. The channel may introduce noise, that is, random distortions or modifications of the message. Also, there may be systematic effects decreasing the information content of the message. Different messages may be received as the same message. This is called redundancy. Redundancy can have the positive effect of error tolerance, in the context of a triplet coding for an amino acid meaning that certain mutations do not affect the amino acid coded for. Indeed, for the receiver, it does not matter which of those different messages have been chosen by the sender as long as the received message remains the same. Thus, the sender can make some errors as long as they do not change the message for the receiver.

In particular, the genetic code is redundant in the sense that the genome as the sender emits nucleotide triplets while the proteome as the receiver obtains amino acids, and several triplets of different chemical composition lead to the same amino acid.5

The application of information theory to molecular biology, however, should go beyond the relationship between individual nucleotide triplets and amino acids. A nucleotide and an amino acid not only have their specific chemical identity, but they are also parts of sequences, the DNA sequence, or a polypeptide chain constituting (part of) a protein. As such, in addition to their chemical composition, they are characterized by their position within that specific sequence. Moreover, the relationship between such a triplet in a specific position and the amino acids coded for by that triplet is not a relationship between individual physical objects, insofar as in a given cell, the triplet is usually expressed several times, and in different polypeptides. Each amino acid produced from the triplet can be considered as a physical instantiation of this particular triplet, and of no other triplet. There are many chemically identical triplets in the DNA, but the given amino acid as a concrete physical object is derived from precisely one such triplet.

Considering it that way, however, falls short of understanding the expression process, and if that were all that information theory can contribute, its usefulness would be rather limited. While in principle we can follow a specific expression pathway and trace the origin of a given amino acid back to a single triplet at its location in the DNA, the formation of that amino acid requires additional ingredients along the expression pathway. Some ingredients come from the cis DNA region containing that triplet. For instance the nucleotide sequence encoded in a promoter region is also needed, and enhancer and repressor sequences affect the expression. Factors in trans, which are specific for the intra- and extracellular environment, also guide the expression. The point in time within the processing sequence also affects the outcome. Thus, the relationship between specific individual chemical units is superseded by processing information that does not implement itself physically in the final product. So, on one hand, when tracing the process back in time, we have a relationship between individual chemical substances determined by their locations within specific sequences, while on the other hand, when going forward in time, we have the combination of cis and trans ingredients determining in which and in how many numbers of polypeptides a given triplet is expressed.

The coding sequence We have four different nucleotides, A, C, G, and T, of which DNA sequences are composed. When each of them occurs with relative frequency p_i (i = A,C,G,T), each position contributes an information of10\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$I_{\text{nuc}}=-\sum\limits_{i=A,C,G,T}{p_{i}\;\text{log}\;p_{i}\; bits}$$\end{document}In particular, when they are equidistributed, that is all p_i = 1/4, this information is 2 (bits). When all positions in a sequence of length N are independent, the sequence information then is I_seq = N I_nuc. Sequence correlations, however, will decrease that information. To make this precise, we need an ensemble of sequences s, and we consider subsequences of length l6 in this ensemble and the block entropies −∑_νp_ν log p_ν (summing over all such subsequences ν of length l, denoting their relative frequencies in our ensemble by p_ν) and let the length l become sufficiently large to capture all such sequence correlations. (In principle, such an analysis is even meaningful when our ensemble consists of a single sequence only, as long as l remains small compared with the sequence length; in any case, the maximal value of l for which the block entropies can be computed in practice is rather small).7We now consider triplets of nucleotides as such triplets are the subsequences coding for amino acids. In particular, when all 64 triplets (including the 2 start/stop codons) are equally frequent (and hence, also the nucleotides are equidistributed), each such triplet contains an amount of I_tri = 3I_nuc = 6 bits of information. There are 20 amino acids out of which polypeptides can be composed; we denote the relative frequency of an amino acid referred to by the index α by p_α. An amino acid thus on average requires for its specification an information of11\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$ I_{\rm aa}=-\sum_{\alpha} p_{\alpha}\;\log p_{\alpha}\;\hbox{bits}. $$\end{document}When all these frequencies are equal, I_aa =  log 20.Due to the degeneracy of the genetic code which leads to redundant coding for amino acids, the information needed to specify an amino acid is smaller than the one contained in a triplet (log 20 < log 64 = 6).Again, sequence correlations will decrease that information. In an ensemble of polypeptide chains, we consider subsequences of length l and the block entropies −∑_σp_σ log p_σ for subsequences σ of length l with relative frequencies p_σ and let the length l become sufficiently large to capture as many sequence correlations as feasible. Again, the maximal value of l for which the block entropies can be effectively computed is rather small.8 When f denotes the relation between triplets and amino acids, that is, f(r) = α when the triplet r codes for the amino acid α, and if we put \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$p(r|\alpha)= \frac{p(r)}{\sum_{\rho:f(\rho)=\alpha}p(\rho)}$$\end{document} (conditional probability for a triplet r given the amino acid α it codes for) when f(r) = α, the mutual information between the collections of individual triplets and amino acids is given by12\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$ I_{\rm {tri,aa}}=I_{\rm tri} - (-\sum_{\alpha} p_{\alpha} (\sum_{r:f(r)=\alpha}p(r|\alpha)\log p(r|\alpha))). $$\end{document}The second term on the right hand side of this equation is the average of a function of the amino acids, where these amino acids are weighted with their relative frequencies. That function is the uncertainty for a given amino acid about the coding triplet. This term thus is the conditional entropy, that is, information, for the triplets given an amino acid, and it quantifies the redundancy of the genetic code. By symmetry of the mutual information (an elementary mathematical result, see Cover (1991)], I_tri,aa = I_aa,tri, the information gained about an amino acid from knowing a triplet. Since a triplet specifies a single type of amino acid, this expression in turn simply equals I_aa, the information contained in an amino acid.Again, since there are sequence correlations, the average information needed to specify a polypeptide consisting of n amino acids is smaller than n I_aa. Thus, also the mutual information between nucleotide and polypeptide sequences will be different from n I_tri,aa.We should point out that here we have computed the mutual information between the chemical compositions of amino acids and polypeptides on one hand and triplets or nucleic acid sequences on the other hand. It is a different question to infer the location of such a triplet in the DNA sequence given the chemical identity of an amino acid or polypeptide.So far, we have presented the standard application of information theory to molecular biology. This, however, is of rather limited use, and we shall now proceed in a different direction, more in line with the general aims of this paper.

Positional information within the coding region When we consider the formation of an amino acid or a polypeptide, not only the chemical identity of the coding triplets is relevant. There are many chemically identical triplets in the DNA, but only one out of those is the origin of a specific given amino acid in a peptide. That triplet can be characterised and distinguished from others by its position in the nucleotide sequence constituting the DNA. This leads us to the information needed to determine that position. The position can for example be described by a coding region or maximal ORF, the sequential number of an exon within that ORF, and the position inside that exon, in an analogous manner as one localizes a word in a book by specifying a chapter, a page within that chapter, and a position on that page.Thus, when considering an individual amino acid, one can quantify the positional information about the location of the triplet coding for it in the DNA. Of course, this information cannot be derived from the triplet in isolation. Not only is the chemical identity of that triplet ambiguous because of the redundancy of the genetic code as analyzed above, but there are also many chemically identical triplets within the DNA. Therefore, the corresponding entropy, that means uncertainty, is rather high.If we use context information, however, the situation changes. The context information comes from the polypeptide chain the amino acid is contained in. Typically, when we know such a polypeptide chain we can uniquely and unambigously identify the coding regions and exons where it is derived from, and inside such an exon, we can then also identify at DNA level the triplet from which our amino acid is expressed. (There exist exceptions to this due to the phenomenon of gene duplication, that is identical genomic regions located at different positions in the DNA coding for the same functional product.) We should point out, however, that we are assuming here that the DNA sequence as such is known and the only uncertainty is about the location of some triplet within that sequence. We have already discussed above, how to quantify the sequence information of the DNA. That information then is assumed to be known in either case considered here, that is, both, when we only know the amino acid in question and want to determine the position of the triplet in the DNA from which it is derived or when, in addition, we have the knowledge about the polypeptide chain containing that amino acid at our disposal. Thus, the alternative is between specifying the position of a triplet in the DNA sequence simply by counting, as described above, or using context information, that is, identifying the polypeptide chain containing the amino acid.9 That latter information will be discussed below. It depends on what class of polypeptide chains the analysis is based. In other words, we need a list of relevant proteins. The information contained in that list, as usual, depends on what we assume as known, for example certain biochemical rules that exclude some amino acid combinations, the species to which the organism in question belongs, or a specific cell type.

The ensemble of products derived from a coding DNA region We now leave behind the standard application of information theory to molecular biology and come to an important issue. In a living cell, from one single coding region or ORF, often a large number of polypeptides is produced, and those may be of different types, because of differential splicing and other regulation processes. Thus, we should not consider the relationship between a single DNA region and a single polypeptides, but rather the one between such a single DNA region and an ensemble of polypeptides. It is here that, in regulation, the program that we are calling the genon enters and provides specific information about the final product from our coding region that is not contained in that coding region itself; the information theoretic analysis should separate these respective contributions. Also, the contributions from the cis and trans programs interact here, and they should then be quantified in information theoretic terms.For evaluating the information provided by the genon, we need to consider the ensemble of polypeptide chains produced under specified conditions, for example those expressed in a given cell or those that can be expressed by the genome in question. Here, when we speak of an ensemble, we always mean a collection of physical objects. These objects may belong to different types, but typically, types are represented by several of such objects, that is, not all of the objects represent different types. Thus, our ensemble consisting of individual physical objects is characterized by the types x to which these objects belong and their relative frequencies p_x with which they occur among these objects (plus an integer for the absolute size of the ensemble, as the p_x are defined as relative frequencies and not as absolute ones). Thus, we consider the ensemble of polypeptide chains produced under some specified conditions, and for each type x of polypeptide represented in the ensemble, we let p_x denote its relative frequency. We compare these relative frequencies p_x of the different types x with the relative frequencies q_x with which they can be derived from the DNA region containing the exons of the coding sequence under consideration. Of course, most q_x will be 0 because any coding region can be expressed only in a small fraction of the polypeptides present in the cell or derivable from the genome. We consider now the difference13\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$ I_{cis}=-\sum_x (p_x \log p_x -q_x \log q_x). $$\end{document}Here, the first term is the uncertainty about a polypeptide when we do not know the coding DNA sequence, whereas the second term, which is negative, that is, subtracted from the first one, quantifies the remaining uncertainty when we already know that coding DNA sequence.In view of the preceding, we expect that I_cis is quite large. In that sense, our coding region encodes a lot of information about functional products. We emphasize once more that this quantity depends on the ensemble (x,p_x). As explained in the introduction, information is measured as a reduction of uncertainty, and therefore, we need to specify first about what there is uncertainty. In principle, we could consider all biochemically possible polypeptides x, even though it might be difficult to assign probabilities p_x to them. That is the situation where we don’t admit any information about the genome or cell in question. We could also be more specific and admit some of the latter information. In that situation, our initial uncertainty is smaller because we already have some knowledge about which polypeptide chains could possibly occur. Therefore, the knowledge of the coding region tells us less that we did not already know than in the previous situation. This issue will again be taken up in the next section.There is one remark of fundamental importance here: while the term −∑_xp_x log p_x in (13) is rather arbitrary because it depends on the choice of the ensemble of possible x, like all combinatorially possible, all chemically possible polypeptides, or all polypeptides occuring in a given organism or cell (an issue to be returned to below), the other term, −∑_xq_x log q_x is not arbitrary at all, because it is derived from the frequencies of the products derived from our coding region under given circumstances. It is this latter term that is important for us and to which we shall turn in the next section.In any case, the reader should note that compared to the beginning where we have discussed the coding information contained in a sequence, we have now completely shifted the perspective. In (13), the contribution of a cis coding region is now a residual term that is obtained by subtracting from an ensemble entropy the contribution of the regulation by genon (and transgenon).

We have quantified the types and numbers of polypeptides derived from a given coding region (genomic domain) by the second term in (13), that is,14\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$ -\sum_x q_x \log q_x. $$\end{document}This information cannot be found in the coding region, but is rather provided by the (proto-, pre-)genon (and the transgenon, a distinction to be addressed below). We now wish to analyze that genon contribution for the transition from the coding region to the gene in terms of information theory. In order to simplify the presentation, we start with a triplet of nucleotides in the DNA and follow its expression path. Along this path, regulation by other factors will determine the fate of the transcripts, i.e., its products, and we shall understand that as an information contribution.

Within the total protein content of a cell, we consider the ensemble of amino acids derived from the given triplet in the DNA. Whereas the chemical structure of these amino acids is the same, their number, that is, the number of copies derived from the same triplet, may vary. In addition, due to differential regulatory effects on the expression pathway, for instance differential splicing, these amino acids may find themselves in structurally different polypeptide chains. The corresponding types we identify by the index x. The information content of this ensemble of polypeptide chains now depends on what we are ready to assume as given.

Before listing some possibilities for quantifying that information content, we recall a general observation from our above discussion of the entropy: When we have a collection of physical objects, we can either list them as such, or we can seek regularities, for example identify types represented by several individual objects, to achieve a more compact representation. In the sequel, we shall begin with the naive list and then proceed to a representation in terms of types x and their relative frequencies p_x.

Explicit description of all physically present polypeptides in the cell containing an amino acid derived from the triplet under consideration. When no further regularities are taken into account, this becomes n₀I_aa where n₀ is the combined length of all these chains.10 Of course, this is only a coarse, and not very helpful, upper estimate of the necessary information. For example, when the ensemble contains several copies, say m, of one particular polypeptide of length n, then the corresponding information can already be described by at most log m +  nI_aa bits instead of the typically much larger number of mnI_aa bits. Also, as already discussed above, we can exploit sequence regularities for the individual polypeptide chains to get below nI_aa bits for such a chain.

In particular, we can then compare the information provided by different DNA domains and thereby specify the information content of the protogenon. Let us consider a sequence s of nucleotides in the DNA, for example again a coding triplet, to start with the smallest unit relevant for the present purpose. As explained above, we have the uncertainty about the ensemble of polypeptides containing a piece of a polypeptide chain, like an amino acid in the case of a triplet, derived from s, expressed through the conditional entropy defined as 17\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$ H(x|s) := -\sum_x q_x \log q_x. $$\end{document}Now, when we know a longer sequence S containing the original s, then we can compute the corresponding quantity H(x|S) (where the x as before stand for those polypeptide chains that contain amino acids derived from s) which now is smaller because the additional information of S\s (the rest of S when s is taken away) now makes more specific predictions of the polypeptide chains possible. The important quantity expressing how much the fate of the products derived from s is constrained by the surrounding region S then is 18\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$ H(x|s)-H(x|S). $$\end{document}Given s, and thus the ensemble determined by (x,q_x), we can then let the surrounding region S vary and detect from (18) the amount of allo-determination of the products derived from s.

We are now in a position to assess the information content of the protogenon. Here, we take as s (fragments of) the coding sequence for some gene, while S is a larger DNA region containing regulatory elements or other protein binding sites that are not part of the coding sequence, introns etc. The expression H(x|s) − H(x|S) from (18) then quantifies the information contribution of the part of the genon captured by S.

When we wish to analyze a specific transgenon and its information contribution, then, instead of adding some further cis elements to the original sequence s, we now take some factors from trans. Then the analysis proceeds as developed above for the cis genon.

In any case, when s again is our coding region, the uncertainty 19\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$ H(x|s) = -\sum_x q(x) \log q(x) $$\end{document}is precisely the amount of information about the products derived from s that comes from outside s, that is from the genon and its precursors, from the transgenon, and from external factors. By varying S, we can then quantify the various individual contributions.

Before proceeding, let us briefly make the following remark: whereas here we have considered the situation for P-genes, the case of R-genes can be handled by the same type of analysis. Futhermore, let us recall that our analysis dealt only with the assembly of sequences during processing and differential splicing, leaving aside all other regulative interventions controlling gene expression in space and time.

There is a different, but somewhat coarser, method of estimating the information provided by the (proto-, pre-) genon. To see this, we consider a polypeptide and look again at the case of the protogenon; we shall ask about all the DNA sites that contributed to its formation, that is, both, the coding triplets and the ones from regulatory regions that guide the process leading to that polypeptide. In "The coding region" above, we have already studied the sequence informations for the polypeptide and the corresponding coding region in the DNA. By the same method, we can then also evaluate the sequence information of non-coding regulatory sites, both in cis and in trans, i.e., either present in the cisgenon or provided by factors from the transgenon. The former include stop codons, enhancer, promoter, repressor sites, introns that play a role in the expression pathway as binding sites for certain proteins, and the like. The relevant part of the holo-transgenon derives from the coding regions for transcription factors and all other proteins regulating or interfering with the expression pathway.

There is a problem with this approach, however. The reason is that many of the regulatory elements, while being specific to a certain degree, need not only affect the polypeptide under consideration, but also interact with the regulation of other polypeptides. Therefore, the simple sum over the sequence entropy of all contributing sites seems to overestimate their specific information content. Putting it another way, when we consider two different polypeptides, we are not allowed to simply add the corresponding sequence entropies because some of the factors may contribute to both of them, leading to an overestimate for the information needed for the two polypeptides. Nevertheless, this approach might be useful in deriving some upper bounds for the information needed to produce a polypeptide.

The contribution of the cis region with its combination of binding oligomotifs consists in a preselection of the possible binding elements at the particular site under consideration, out of all the proteins in the cell that can bind to DNA or RNA. We first consider one particular site s in cis, and assume for the moment that precisely one protein can bind at that site. Let p_y denote the relative frequency of the RNA or DNA binding protein y in the cell, and let q_y be the relative affinity of y at the site under consideration. For most y, q_y will be 0, because binding requires a special affinity to the site in question. We consider thus the quantitiy 20\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$ I_s := -\sum_y p_y \log p_y +\sum_y q_y \log q_y $$\end{document}where the first term represents the uncertainty about a protein in the cell without knowing the binding site, whereas the second term, which again is negative, that is, subtracted from the first one, represents the remaining uncertainty when we know the binding site, that is, when we only consider those proteins that could possibly bind at that site and their binding affinities. In view of the preceding, I_s is expected to be rather large, and this expression quantifies the specificity of the site.

The basic case from which to start thinking about the genon is where the whole expression pathway is solely controlled by cis, in the sense that all necessary factors are provided by the program represented by the transgenon, and any specificity is entirely due to selection by cis of binding factors. In that case, all q_y = 0 or 1, and (20) becomes 21\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$ I_s = -\sum_y p_y \log p_y. $$\end{document}We have thus considered the most elementary situation. When it comes to processes of differential expression, for example in response to external signals, the ensemble of trans factors becomes variable, and therefore, we need to assign non-trivial q_y to some factors, and so we are back to (20).

Still, this needs to be expanded in two directions. First, a cis region can, and typically does, contain more than one protein binding site. When the binding properties of these sites are independent of each other, we can simply sum the expression given in (20) over all those sites, to get the process information content of that cis region. Such an independence holds when one only considers linearly RNA binding polypeptides.

In other situations, we need to modify this expression by taking correlations into account as in the previous sections. Second, there is an important combinatorial aspect because at one site, usually not a single polypeptide is binding, but some combination of such polypeptides that then biochemically form a quartenary protein complex. Furthermore, some other proteins facilitate or inhibit the binding of certain other ones. Therefore, instead of single proteins, we need to consider protein combinations, as in a language where instead of individual phonems or letters, one rather takes morphems or words as basic elements. The principle expressed in (20) still applies when one substitutes protein combinations for isolated proteins.

In summary, the process information content of a cis region is quantified by a reduction of possibilities. Therefore, it cannot be computed directly from the nucleotides forming the region, but rather depends on the proteome in the cell. This may seem paradoxical, namely that we cannot compute the information contribution of a DNA region by looking at the nucleotides, but rather need to compare the number of possible binding proteins with the smaller number of those actually capable of binding to that particular region. Of course, it is determined by the latter’s nucleotides which proteins can bind there and which ones can’t, but in order to do the computation we need to know which trans factors are the candidates.

Conversely, the information contribution coming from trans simply consists in the selection of those factors that actually bind to a given (proto/pre)genon, out of those possibilities allowed by the structure of the signals in the DNA domain as composed by its nucleotides. Thus, here the difference is between those that can possibly bind, given the concrete nucleotides, and those that are actually provided by the holo-transgenon of the given cell. Returning to (20), the uncertainty left after evaluating the information provided by cis is the term −∑_yq_y log q_y incorporating the affinities (or the corresponding expression taking into account all the binding sites of a given cis region and their combinatorics, that is, the combination rules for the binding of several different trans factors at neighboring or otherwise related sites). The cis region allows certain proteins to bind, but it does not completely specify which ones will actually be bound. That selection is the important trans contribution that constitutes the regulation process. Therefore, when a particular protein has bound to a particular site, that site has gained an information −∑_yq_ylogq_y. According to our information theoretical scheme, that information is not assigned to the cis site, but rather considered as provided by the transgenon. In the terminology of information theory, cis here is considered as the receiver for a message sent by trans, and that message consists in the specification of the binding protein.