Punching Through on Computer Science with Mike Dunlavey

I have further conversations with Mike Dunlavey about various topics in computer science, including information theory, AI, knowledge representation, languages, and macros.

Transcript

The following transcript was automatically generated by an algorithm.

00:00 : i'm curious about like the um
00:03 : origin of your method how did that come
00:06 : about
00:08 : uh well i'm trying to think when i first
00:12 : did that and i think it might have gone
00:13 : back to the
00:16 : 60s maybe when i was i got a
00:20 : consulting job with raytheon raytheon
00:23 : used to do
00:24 : airline reservation systems
00:28 : and they had they had what in those days
00:32 : was called a mini computer
00:34 : it would it would it was a it would fit
00:38 : in a rack
00:39 : that would go against a wall and you
00:42 : program the thing in assembly language
00:44 : and you know we would we would um
00:48 : check to see if our programs were right
00:50 : by single stepping
00:52 : through because you could push a button
00:53 : on the front of the mini computer
00:56 : to single step one instruction and you
00:58 : go step by step to make sure it's doing
01:01 : the right thing
01:02 : and then you get tired of well first of
01:06 : all
01:06 : if you're doing that then if there's
01:08 : anything wasting time you're gonna know
01:10 : because you're stepping through it
01:12 : wasting time and wasting your own time
01:14 : when you want to have lunch
01:16 : okay but so sometimes you get kind of
01:18 : lazy and you hit the rim button and then
01:20 : you hit the halt button
01:23 : oh yeah yeah and then you say well what
01:26 : the heck is going on
01:27 : now and so you see you look at the stack
01:31 : and so on except program computers
01:34 : didn't have stacks in those days
01:36 : but
01:41 : that's going way back when we didn't
01:44 : even have stacks
01:45 : did you sort of um come up with your own
01:48 : way
01:49 : of doing function calls or sort of a
01:53 : simulating a stack or something using
01:56 : no i think st actual program stacks
02:01 : see back in the in the original fortran
02:03 : days
02:04 : if you had a fortran routine that if you
02:07 : had fortran
02:08 : routine a that called b that called a
02:12 : you could tell because the lights on the
02:14 : front of the computer would go blank
02:16 : because the thing was in a loop it was
02:18 : it was an infinite loop
02:20 : because it was returning to itself you
02:22 : know that uh digital
02:24 : pdp eights or pdp tens i forget
02:28 : came along and they actually put a stack
02:30 : into the
02:31 : hardware so that routine a could call
02:34 : routine b
02:35 : and which could call routine a and when
02:37 : a returned it would come back to b
02:40 : if i feel like i sometimes do that when
02:43 : i'm
02:43 : my stack is too full instead of saying
02:46 : stack overflow i just forget the top
02:49 : parts of the stack
02:52 : i i digress so many times i forgot what
02:55 : i was trying to do in the first place
02:57 : in those days there was fortran there
02:59 : was algol there was
03:02 : snowball and somebody along the way
03:05 : said we want to have functions that can
03:07 : call themselves like uh
03:09 : do a factorial okay well you gotta have
03:13 : a stack
03:14 : and so they said well okay we'll in our
03:16 : in our uh
03:18 : compiler we'll have it put in stack
03:20 : we'll have it
03:21 : make its own stack hard to imagine
03:24 : a time when we we didn't even know how
03:27 : to do function calls that's
03:29 : we're talking you know the late 50s
03:33 : knowing my age here in those days um
03:36 : you know recursion would turn out to be
03:39 : a big deal
03:40 : but then along came object-oriented
03:42 : programming
03:43 : and people thought oh this is so great
03:45 : let's teach everybody this
03:48 : yeah and uh i was in college back then i
03:50 : was at georgia tech yeah
03:53 : and then everybody was on the java train
03:55 : you know
03:56 : yeah yeah yeah yeah so go figure
04:03 : um so you uh you imagine you wrote a
04:06 : book
04:07 : um and it's it's called like
04:10 : making making programs better or
04:13 : something or making
04:14 : programs building there's a stupidness
04:16 : of stupid titles
04:18 : building better applications now when i
04:20 : read it
04:21 : i cringe but um
04:24 : the thing is that performance tuning was
04:27 : only just one
04:28 : little part of it you started with the
04:31 : um performance tuning part and then
04:34 : does that sort of shine a light on other
04:37 : aspects
04:38 : of programming well if if you look at
04:42 : algorithms like like um you know table
04:45 : lookup
04:47 : um you've you've learned various ways to
04:50 : do it
04:51 : and um there's a nice there's something
04:55 : called information
04:56 : theory shannon information theory uh
04:59 : which
04:59 : is just kind of like the original
05:02 : information theory used in coding and
05:05 : telephones and things
05:08 : and it does a beautiful job of
05:11 : explaining
05:12 : and accounting for the performance of
05:14 : these various algorithms
05:17 : but there's more than one information
05:19 : theory
05:20 : there is something called komogorov
05:22 : information theory
05:24 : where instead of say having
05:28 : a set of symbols like zero and one
05:32 : you have programs programs say
05:36 : written for touring machines
05:40 : and those programs themselves
05:43 : have a concept of information associated
05:46 : with them
05:47 : you can say you can say for instance
05:50 : that the suppose you have an arbitrary
05:53 : string
05:54 : you can see that the information content
05:57 : of that
05:57 : string is the length of the shortest
06:00 : program that can print it
06:01 : the information of that string
06:04 : i would imagine to print any string
06:07 : you just loop through the characters and
06:10 : print them out
06:11 : one by one right well then the
06:14 : characters themselves have to be
06:15 : somewhere
06:16 : oh that i see the characters themselves
06:19 : have to
06:20 : be stored in like static memory usually
06:24 : yeah you have to take the um
06:27 : you know you can think of all kinds of i
06:29 : i i got it yeah
06:31 : i see i see what you're saying that
06:33 : you're saying like
06:34 : if you're just printing out for example
06:37 : uh
06:38 : the string one two three four five six
06:39 : seven eight nine ten well
06:41 : you don't have to store all of that
06:43 : because you can calculate it or if
06:45 : if your output is like the fibonacci
06:49 : sequence
06:49 : or something like that then you can
06:52 : calculate you can compute the infinite
06:54 : series
06:55 : there's a short there's a short program
06:57 : and since there's a short program that
06:59 : can print it
07:00 : it's information content is no more than
07:02 : that got it
07:03 : yeah you know that you know it's bigger
07:06 : than one bit
07:07 : because there's no program one bit long
07:10 : that can print that stuff
07:11 : you've done it yeah okay
07:15 : and then you know i i spent
07:18 : a good part of the 70s in the ai lab
07:21 : and um there the idea is your brain is a
07:24 : computer
07:25 : and you have ideas in your head and the
07:29 : ideas in your head are like data
07:30 : structures
07:32 : but those those ideas are expressed
07:36 : a data structure is a language because
07:38 : it contains
07:40 : it contains terms that express concepts
07:45 : and so the idea is that
07:49 : your people are really good at finding
07:54 : good concise ways to express the ideas
07:57 : that are in their head
07:59 : and when they communicate them to
08:02 : another person
08:03 : they pick words and nouns and verbs and
08:06 : adjectives
08:07 : that that convey the ideas in your head
08:11 : and that that language that they're used
08:14 : they're used to express it
08:16 : is is related is closely related to
08:19 : the terms in their head that they use to
08:21 : think about the problem
08:25 : okay so so
08:29 : what what i look for in a programming
08:31 : language
08:32 : is that i'd like to be able to tell the
08:35 : computer
08:37 : just as i can tell another person in a
08:41 : language that's
08:42 : that that closely matches what i'm
08:45 : what i'm thinking so that if i change my
08:49 : mind
08:50 : if i make a small change in my mind that
08:53 : could
08:53 : that should result in a small change to
08:56 : the program that i'm typing
09:00 : i see yeah okay and you can measure that
09:04 : by means of
09:05 : you can use a diff program i see
09:08 : it it's kind of it relates very much to
09:11 : even though it
09:12 : sounds very theoretical but it's i would
09:15 : say
09:15 : very much related to practical uh
09:19 : work because because you don't
09:22 : want to have to make huge
09:26 : sweeping changes to your code base every
09:28 : time you have to
09:30 : make a modification to your program
09:33 : yeah you want to be you want to be and
09:36 : and that's in a way that's the thing
09:38 : that's challenging about
09:40 : work in the industry is is like
09:45 : how what is the correct way to structure
09:48 : your program
09:50 : because we want the program to be
09:52 : maintenance
09:53 : maintainable in the long run and
09:56 : maintainable means
09:58 : for any given feature that we have to
10:01 : add to the program uh we we hope to make
10:05 : the
10:05 : least a small change to it to make that
10:08 : happen
10:10 : that's that's exactly right and
10:14 : um people are not taught to do that
10:17 : and and the result is you know like
10:21 : the the company i just left they have
10:25 : mega lines of c plus plus
10:29 : implementing things that could be done
10:30 : in in you know the ideas in it
10:33 : can be expressed in in
10:37 : uh academic papers they can write us
10:40 : they can write a single equation you
10:42 : know saying
10:44 : some sum of such and so and uh
10:47 : and it shouldn't be it shouldn't be less
10:51 : than 10 times that much
10:54 : in terms of programming code to
10:56 : accomplish it but it is its way it's a
10:58 : thousand times
11:00 : so um so the slightest
11:04 : change is something that has to go on
11:06 : the
11:07 : on the list of things to be done and it
11:11 : can take
11:12 : months to do it and so on
11:16 : yeah the thing is if i can continue
11:20 : for a second you know so that's why i
11:22 : think designing languages
11:24 : is really important when you're
11:25 : designing a data structure you're
11:27 : designing a language
11:29 : because you're deciding what the terms
11:31 : are yeah and
11:32 : how they're connected and you know there
11:36 : are many ways to implement languages one
11:38 : of my favorites is just use the
11:40 : preprocessor the cp
11:41 : processor now in java they decided that
11:45 : was a bad
11:45 : idea you're talking about macros
11:50 : yeah macros you can
11:54 : you can have macros that do all the
11:56 : things that you need done and so you can
11:58 : write your program using the macros and
12:00 : it will be a ton
12:01 : shorter and easier that was hilarious
12:05 : yeah that's how i know the lisp
12:07 : community is really into that they they
12:09 : even
12:10 : formalize it a bit more probably
12:13 : because their their macro generators can
12:16 : read the ast
12:17 : and so forth you got it you got it
12:21 : yeah i i heard the list was called lots
12:23 : of irritating single parentheses
12:26 : that's true that's true they just i i
12:29 : just
12:30 : yeah it's just too many too many to have
12:33 : to match up
12:35 : but but you never have to deal with
12:37 : parsing because
12:38 : it's it's automatic that's true yeah
12:41 : that's a
12:41 : really interesting actually i would if
12:44 : you don't
12:45 : mind go ahead because i'm kind of a noob
12:48 : to uh to see programming oh i did a
12:51 : little bit in college but that's so many
12:53 : years ago
12:54 : um so now that i'm
12:57 : doing more work in c again
13:01 : like usually i stayed away from
13:05 : uh macros because i i thought
13:08 : that well that that's kind of dark magic
13:12 : kind of thing and it's kind of hard to
13:14 : debug if you make a mistake
13:16 : it's it's harder to uh diagnose and so
13:19 : forth
13:20 : um so i mostly stayed away from it but
13:23 : recently i made a use of it i try to use
13:27 : it in the context of
13:28 : error handling
13:32 : i don't say go bananas with with macros
13:36 : i'm saying use them carefully
13:39 : so so i kind of want to get your opinion
13:42 : on
13:43 : a scheme that i came up with
13:46 : for uh for error handling in
13:49 : c oh i think it's this one
13:53 : and see if you think it's or
13:57 : this your opinion on it and what do you
14:00 : have suggestions for me
14:04 : okay so this is my idea so so the
14:07 : problem
14:08 : like that i wanted to solve was the fact
14:11 : that
14:12 : like any time you call a function that
14:15 : could potentially have
14:16 : an error the standard thing to do in c
14:19 : is like you return a numeric error code
14:23 : and uh and then you expect the caller to
14:26 : have to check
14:27 : the error code and then
14:30 : and the problem with that is i find
14:32 : myself writing
14:34 : so many of these if statements and then
14:37 : a little bit of error handling
14:39 : over and over again and since i'm coming
14:42 : from
14:43 : you know higher level languages like
14:45 : python
14:46 : i kind of wanted to have like a
14:49 : cleaner or more terser syntax
14:52 : in general so i i was inspired by
14:56 : one of the some something i found on the
14:58 : internet
14:59 : where where when i call a function
15:03 : i wrap it with a macro which would do
15:07 : a little bit of that error checking and
15:09 : then
15:11 : and then and then
15:14 : in the event that in the event there's
15:17 : an
15:17 : error code it just return one
15:20 : assuming we're within the context where
15:23 : the function also is going to return an
15:26 : error code
15:26 : so it sort of just passes on the error
15:29 : code
15:30 : uh to the to the current function
15:34 : and and so i i get to write little bits
15:36 : of code
15:38 : like this and then in my larger program
15:40 : i've been making use of it
15:42 : it looks like this where like each each
15:45 : one of these steps could
15:47 : have an error code and i need to do a
15:49 : little check and then do some error
15:51 : handling and now i just wrap it up with
15:53 : this macro that
15:54 : if there's an error that's return one to
15:57 : signal that there's an
15:58 : error for our function as well
16:01 : pass it on past pass the book and i feel
16:04 : like that's sort of like a poor man's
16:06 : exception handling in a way what do you
16:09 : think of this idea
16:11 : i think is a great idea one of the
16:14 : things you have to
16:16 : face when you do things like this
16:19 : is that you impose a learning curve on
16:22 : the next person who comes after you
16:25 : and my point is that
16:28 : that's not a bad thing
16:31 : in other words if if you can
16:34 : if you can shrink your code by a factor
16:37 : of 10
16:38 : and still keep it quite readable
16:41 : that's worth asking your the person who
16:43 : comes after you
16:44 : to learn something about it it's because
16:47 : um often you you get you know you get in
16:51 : the in the programmer
16:53 : uh rules you will say
16:57 : well it shouldn't anybody should be able
16:59 : to come up and step
17:01 : up and and uh look at it and work on it
17:05 : and i'm saying well you know sorry but
17:08 : um we programmers are not monkeys
17:12 : they have to learn
17:16 : and and it's worth spending the time
17:19 : to to educate them yeah
17:22 : and then and part of what you do in the
17:24 : code is you should
17:26 : should speak to the programmers that
17:29 : come after you and say this is how you
17:30 : do it
17:32 : yeah yeah that that kind of issue has
17:35 : been
17:37 : you know it's been come comes up and
17:40 : we talk about it at work quite often of
17:43 : like
17:44 : we're introducing these sort of smart
17:46 : ways of doing things
17:48 : but then now it's gonna make it harder
17:50 : for everyone to learn it
17:52 : um there's a balance to strike i think
17:56 : uh but yeah like when you're doing
17:58 : clever things like this
18:00 : you do you should think about
18:03 : okay how am i gonna get the other folks
18:06 : on board here
18:07 : what is a good way what's a smooth
18:10 : learning experience for them that's
18:12 : where you're teaching experience comes
18:14 : in handy
18:15 : because you're teaching them
18:20 : yeah yeah for sure for sure
18:23 : yep thank you so much for your feedback
18:26 : this has been a lot of fun for me and
18:30 : i'm glad i was able to show you some of
18:32 : my code
18:33 : and uh appreciate i appreciate the
18:36 : knowledge that you imparted on me and on
18:40 : other folks on stack overflow i can see
18:42 : well i appreciate
18:44 : i appreciate your interest yeah
18:47 : for sure yeah it's it's really really
18:50 : important technique i think i think um
18:54 : it almost feels like something that all
18:57 : programmers
18:57 : should know but for some reason most of
19:00 : them
19:01 : don't so yeah i i hope more people will
19:05 : learn about your technique
19:06 : and try it out and i i might i might do
19:10 : more videos about it like
19:11 : just show that i see it as as
19:15 : punching through on computer science
19:18 : because it's
19:18 : computer science tends to get stuck in
19:21 : groups
19:22 : and this is a way of getting out of that
19:25 : yeah yeah yeah get get get into these uh
19:28 : echo chambers right yeah
19:33 : yeah let's just try try to get out of
19:36 : your own
19:37 : echo chamber whichever one that is
19:41 : i i think it's healthy to uh
19:45 : to hop from one programming community or
19:49 : to another
19:50 : and then to another and then perhaps
19:53 : you will see more of the world that way
19:57 : all right yeah learn learn more
19:59 : different programming languages
20:02 : thank you thank you so much