From ResearchID.org

Contents 1 Information heirarchy 2 Information 3 Information entropy 4 Leibniz 5 Algorithmic Information Theory 6 References 7 Type  (?) 8 Level  (?) 9 Definition  (?) 10 Objectives  (?) 11 Questions  (?) 12 Keywords  (?) 13 Monographs  (?) 14 eResources  (?) 15 Criticism  (?) 16 Status  (?) 17 Extend  (?) 18 Related  (?) 19 Notes  (?) 20 Field(s)  (?)

This page about Information Theory is a stub. Please help us make this article by contributing the missing information.

Introduction  (?)
Information theory is the study of data with specified messages or meaning. Claude Elwood Shannon was the creator of this theory. Information Theory regards information as only those symbols that are uncertain to the receiver in communication. For years, people have sent telegraph messages, leaving out non-essential words such as "a" and "the." In the same vein, predictable symbols can be left out, like in the sentence, "only infrmatn esentil to understandn mst b tranmitd." Shannon made clear that uncertainty is the very commodity of communication. The amount of information, or uncertainty, output by an information source is a measure of its entropy. In turn, a source's entropy determines the amount of bits per symbol required to encode the source's information. * from the belllabs website

Information heirarchy

Werner Gitt categorized information into a heirarchy with five levels: Gitt categorized information into five levels:

1. Statistics: Symbol frequencies, channel capacity etc. See: Information Entropy, Shannon's Theory
2. Syntax: All structural properties of setting up information.
3. Semantics: Meaning of symbols.
4. Pragmatics: Actions required by recipient to achieve sender's purposes.
5. Apobetics: Sender's purposes.

Gitt formulated thirty two empirical Gitt's Information Theorems to detail his information heirarchy. Probabilities

Entropies

Information

• Information: Wikipedia

Information entropy

• Information Entropy: Wikipedia
• Information Entropy: Answers.com

Bayesian

Leibniz

Leibniz asserts that God simultaneously maximizes the variety, diversity and richness of the world, and minimizes the conceptual complexity of the set of ideas that determine the world.^[1]

Algorithmic Information Theory

George Chaitin has published Meta Math (2005) in which he develops algorithmic information theory. In his Alan Turing [Lecture 2005 ^[2] George Chaitin summarized:

program (bit string) ---> Comptuer ---> output (bit string)

theory ---> Computer ---> mathematical or scientific facts

Understanding is compression!

A theory, an explanation, is only successful to the extent to which it compresses teh number ofbits in the facts into a much smaller number of bits of theory. Understanding is compression, comprehensionis compression! That's how we can tell the difference between real theories and ad hoc theories.

References

1. ↑ cited by: G. Chaitin, Epistemology as Information Theory: From Leibniz to Omega. 2005.
2. ↑ G. Chaitin, Epistemology as Information Theory: From Leibniz to Omega. 2005

Type  (?)

Theoria, Praxis

Level  (?)

ALL (Undergraduate, Graduate, Doctorate, Postdoctorate)

Definition  (?)

--- ! Begin definition instructions ! ---

• This section could include information about:
  • types of activities that may be involved,
  • what kinds of resources to use for research,
  • structured research plans,
  • what lab tools will be valuable, or
  • identify and define research terms.
• (700 characters or less)

--- ! End definition instructions ! ---

Objectives  (?)

--- ! Begin objectives instructions ! ---

• Identify clear research goals. These could include definite goals about data collection, observation, or desired outcomes of theoretical research. (700 characters or less)

--- ! End objectives instructions ! ---

Questions  (?)

--- ! Begin questions instructions ! ---

• List 1-7 questions to be explored.
• If a research application has more than 7 questions, it may be possible to split some of the questions into a new research article.

--- ! End questions instructions ! ---

Keywords  (?)

information, shannon information, uncertainty, entropy, bayesian information

Monographs  (?)

The Design Revolution, William Dembski, ISBN 0830832165
Chapter 17

eResources  (?)

--- ! Begin eResources instructions ! ---

• Resources on the Internet that are useful for studying this topic. A hyperlink, short description, and the full URL. Please use this format:
  

Descriptive Name with Embedded Hyperlink (Website content description)
http://www.researchid.org

• (1-10 links)

--- ! End eResources instructions ! ---

Criticism  (?)

--- ! Begin criticism instructions ! ---

• The most useful critiques of this research application, included here as a simple title with a hyperlink or citation.
• This section is not for the inclusion of a critical abstract, only links or citations.
• (1-10 links or citations)

--- ! End criticism instructions ! ---

Status  (?)

Current William Dembski, ISCID

Extend  (?)

--- ! Begin extend instructions ! ---

• Other questions that follow from this application that would be worthy of investigation.
• Entries in the 'Extend' section could have high potential for becoming new research applications.
• (700 characters or less)

--- ! End extend instructions ! ---

Related  (?)

Bioinformation, Bioinformatics

Notes  (?)

--- ! Begin notes instructions ! ---

• Use the citation and notation style of choice for the field, or the style that you would prefer use. Consistency within the article is the goal. If you want some ideas on style, check out Wikipedia's citation style examples.
• How to use "wikified" citations.

In the text, type in:

 {{ref|descriptionname}} 

In the "Notes" section, type in:

 # {{note|descriptionname}} 

• All of the notes under this section should be surrounded by these "div style" tags in order to make the text smaller:

<div style="font-size: 85%"> </div>

• Be sure to remove the nowiki tags to allow the div tags to take effect.

--- ! End notes instructions ! ---

Field(s)  (?)