Thursday, June 23, 2016

Evolution of my library

Back in 1973, when we lived in a Government Bungalow called Canalvilla, situated at princely location on the corner next to Chitpur Bridge, my books, all being school textbooks, were accommodated in a small wooden bookshelf. Behind the books, I used the keep stuffs like storybooks hidden from the eyes of my parents.

In that year, we shifted to our new home at Ballygunge Place and next year I joined South Point. Sometimes in April/May, I visited Anjan Babu at his home….his bookshelves full of Physics books kind of started a simmering desire to have lot of books. Physics was the subject I fell in love..and Anjan Babu introduce me to the world of foreign authors…Gamow: Physics Foundations and Frontiers, Isaac Asimov: Understanding Physics, Greene: Physics, White: College Physics. With interest grew my urge to buy books…it was in Class X and XI, I bought Glasstone: Physical Chemistry, AJ Mee: Physical Chemistry, Piskunov, Berkley Physics Course Vol I at a princely sum of Rs 10.75, Resnick and Halliday, both the volumes,Scientific American Compiled Papers 3 volumes of Physical Science…so the bookshelf was looking respectable. Along with that grew my habit of scoring the railings of Presidency College, the bookshop on Dover Lane, Oxford (those days they kept both science , mathematics and Engineering books).

By the end of the fourth year and in final year of IIT Kharagpur, I had fallen in love second time: with Digital Communication. That was the beginning of my Digital Communication library…with Taub and Carlson, then Communication Circuits by Clarke and Hess, Jayant: Waveform quantization and Coding, IEEE. I got Dixon's Spread Spectrum Systems IEEE, Ash: Information Theory as a gift from an aunt who was a resident of USA.

By the time I was due to leave for Academy to join IAS, my library has grown to a respectable one almirah and one bookshelf full of books, about 40% from the various subjects of B.Tech curricula like Millman-Halkias, Millman-Taub, Taub-Schilling etc but along with that books on Physics (my first engagement with Civil Services Examination) and Mathematics (my second and final engagement).

My job kept me away from my quest of books and learning but in 1994, when I came back to Calcutta and joined WBSEB, I had brought back from Balurghat a Science and Technology encyclopedia. In WBSEB, amongst engineers many of whom were academically excellent people, my old passion for communication theory got rekindled and my library started growing in size. By that time, I have been allotted the mezzanine floor room for my library and my parents even sponsored a couple of very decent bookshelves to be built for keeping my books.

It was around this time probably in 1995, I met Arabindada of Dasgupta Publishers, who became a constant encouragement and supplier of books. He got me my first copy of Courant's Calculus and Wyl's Space, Matter and Time, max Born's Atomic Physics and umpteen books on Digital Communication and Spread spectrum. On one trip to Hyderabad, I went to a bookshop where I almost went crazy: so many books which I have only read about in the bibliographies… I borrowed some money from my fellow colleagues and bought a suitcase full. Those days in Hyderabad airport one had to walk on the tarmac from the terminal to the craft and while I was walking, the straps of my suitcase weighing several kilos broke; oops….dragged the beast somewhere.

During my tenure in KMC, I first bought books from Amazon. One librarian in British Council where I have gone on some program and landed in the library searching for Delta Modulation by Steele, told me about Abebooks.com. Money was problem; my father being a strict disciplinarian and manager of my money ( YES!!: He managed my money and I was only given a pocket expense when I was controlling crores of government money as I didn't have the time and inclination to manage anything other than my job) will not allow my book expense beyond a point. Not that the salary was so princely as to afford matching my hunger for books….but nevertheless I managed to buy books at regular frequency.  The two bookshelves were no more sufficient so came third, fourth and the fifth…

Oh! Sometimes in 1999, I was told by a friend of mine working for Prentice Hall of India that they have marked more than 5000 books for destruction (yes destruction…they tear off the title page and the book as per their accounting standards become valueless!!!) and these books are stored in the garage of their MD at his house in Bansdroni. One saturday I went down there, it was like two garages and the books were stacked without any space to even walk between the stacks….so I took off my shoes and spent some four hours going through rummaging the stacks and had about 150 books. They were loaded in the boot of my ambassador staff car, the car's backside sagged; I had to give up loading when my driver told me that the car could not be driven if more are loaded….what books my God!!! Rare, relevant, hardcover, original US edition …..its a treasure trove.

In 2007, when I was about to leave government service, my parents agreed after a lot of cajoling that the library has become much bigger to be accommodated in the mezzanine floor room and books need more space in house. So the first floor flat was designated to be the library. with some more new shelves the library in its current form started blossoming….years went by….9 years….more books, more interest in Mathematics and well I have now shortage of space…yes Shortage….every conceivable wallspace has now bookshelves and there is no more wall….I need some more space for my books….

Tuesday, June 21, 2016

Bregman Divergence

Learnt a new mathematical concept: Bregman divergence. Knew about Kullback-Leibler divergence but this one I didn't know about. The concept of dual space is very interesting.  the following is from Wiki page.

Definition

Let $F\mathrm{\Omega }\mathbb{R}$ be a continuously-differentiable real-valued and strictly convex function defined on a closed convex set $\mathrm{\Omega }$.
The Bregman distance associated with F for points $pq\mathrm{\Omega }$ is the difference between the value of F at point p and the value of the first-order Taylor expansion of Faround point q evaluated at point p:
${D}_{F}$ p qF pF qF q pq

Properties

• Non-negativity${D}_{F}$ p q0 for all p, q. This is a consequence of the convexity of F.
• Convexity:${D}_{F}$ p q is convex in its first argument, but not necessarily in the second argument (see [1])
• Linearity: If we think of the Bregman distance as an operator on the function F, then it is linear with respect to non-negative coefficients. In other words, for ${F}_{1}{F}_{2}$strictly convex and differentiable, and $\lambda 0$,
${D}_{{F}_{1}\lambda {F}_{2}}$ p  q  DF1 p  q  λ DF2 p  q
• Duality: The function F has a convex conjugate ${F}^{}$. The Bregman distance defined with respect to ${F}^{}$ has an interesting relationship to ${D}_{F}$ p q
${D}_{{F}^{}}{p}^{}{q}^{}$ DF q p
Here, ${p}^{}\mathrm{\nabla }Fp$ and ${q}^{}\mathrm{\nabla }Fq$ are the dual points corresponding to p and q.
• Mean as minimizer: A key result about Bregman divergences is that, given a random vector, the mean vector minimizes the expected Bregman divergence from the random vector. This result generalizes the textbook result that the mean of a set minimizes total squared error to elements in the set. This result was proved for the vector case by (Banerjee et al. 2005), and extended to the case of functions/distributions by (Frigyik et al. 2008). This result is important because it further justifies using a mean as a representative of a random set, particularly in Bayesian estimation.

Examples

• Squared Euclidean distance ${D}_{F}$ x yxy2 is the canonical example of a Bregman distance, generated by the convex function $Fxx{}^{2}$
• The squared Mahalanobis distance${D}_{F}$ x y12 xyTQ xy  which is generated by the convex function $Fx\frac{1}{2}{x}^{T}Qx$. This can be thought of as a generalization of the above squared Euclidean distance.
• The generalized Kullback–Leibler divergence
${D}_{F}$ p  q  p i logpiqi p  i  q i
is generated by the convex function
$F$ p ip i logp i p i
${D}_{F}$  p  q   ipiqilogpiqi1
is generated by the convex function
$F$ p logp i

Friday, June 17, 2016

Marriage of two Completely different ideas

It was alate afternoon and I was studying Differential Geometry of Manifolds. I was smoking a cigarette and just simply introspecting on what I had just learnt: Concept of charts and local neighborhoods on a manifold. A tiny idea started idea gradually germinated in my mind…an n-dimensional coordinate space and in communication we have n-dimensional signal space …can they be similar or rather can there be some kind of cross-germination…Next time I was on my computer, I did what everyone does….Google….a little bit search and I come across the name of a gentleman called Shunuchi Amari who has written few papers back in 1960s on geometric analysis of communication system… frantic search in the net but these papers are not available. Email to Prof. Amari and in 36 hours his assistant sends me a treasure trove….four papers going in the same direction as I had thought and An Idea was born…..

Today thats what is the subject of my Doctoral Research…it is somewhat uncharted territory but I have a lighthouse…a subject called Information Geometry…Prof Amari can be called the father of the subject which is a beacon in my journey….may be that will make me reach my destination.

It has been a long time…..

It has been a long time since my last post. In the meantime, I have fallen in love….with Differential Geometry. I don't know how it happened….but it has. I have also acquired my M.Sc degree in Pure Mathematics..so logically can call myself a Mathematician now along with a Digital Communication Engineer. My two years sojourn into the world of structured study of Mathematics while studying for the 20 papers that M.SC degree entailed made me learn a lot of things which left to myself I would have never learned…Topology, Topological Vector Space, Graph Theory and believe me…C Programming and Numerical Analysis. They are all wonderful things but somehow on my own I would not have made myself to learn it unless forced which happened.
Writing the exam was also a great fun. The weather was sultry, mid July to mid September, No air-conditioning, sweaty hands, supervisory eye of the invigilator, small benches which now my backside hardly fitted…..a sense of deja vu prevailed. Weirdest part was being called by my co-examinees "Uncle"…augh…oops…trimmed my mustache so that the whites show their minimum, took deep breaths to suck in the paunch but to no avail…."Uncle…"Huh ..! I later figured out that I was also an enigma to them…I am not a teacher, nor I am looking for job of a teacher then what the hell am I? They wondered..Nobody could imagine that I am so crazy a nut that I am doing my M.SC which I need not do..nothing forcing me to go through the grueling process..but I was doing it..and It was great fun.