Skip to content

10




文本

████    重点词汇
████    难点词汇
████    生僻词
████    词组 & 惯用语

[学习本文需要基础词汇量：6,000 ]
[本次分析采用基础词汇量：6,000 ]

So hello everyone, and welcome for the last lecture of CS230 Deep Learning.

So it's been ten weeks and you've been,

you've been studying deep learning all around starting with fully connected networks, uh,

understanding how to boost these networks and make them better and then, uh,

using recurrent neural networks in the last part

as - and convolutional neural networks in the fourth part,

uh, to build models for imaging and text and other applications.

So today's class wrap-up and, uh,

the lecture might be, uh,

slightly, um, shorter than usual.

But we're going to go over a small case study

on conversational assistants to start with which is an industrial topic.

Um, we will do a small quiz competition with Menti,

and the fastest person who has the best answer will

win 400 hours of GPU credits on Amazon.

[LAUGHTER] So you guys can,

can start, can start working on it.

Um, we will see some class project advice because you guys have about two weeks,

uh, less than two weeks before to post your presentation,

and the final project due date.

We'll also go over some of the next steps after CS230,

what have our students done over the past year and

what we think are good next steps and closing remarks to finish.

Oh by the way, if you have a Clicker with, with batch would you please,

can you bring it to me. Um, okay.

So let's get started with how to build a chatbot to help students

find or/and enroll in the right course.

So this is going to be a pretty simple case of a chatbot because chatbots and

convertional - conversational assistants in

general have been very hard to build and have been an industrial topic.

There are some places where academia has helped,

uh, the chatbot improvement.

And here we're going to see how we can take all our algorithms,

what we've learned in this class and plug it in in a conversational setting.

That sounds good?

So let me give you an example,

students might write to the chatbot, "Hi,

I want to enroll in CS106A for Winter 2019 to learn coding."

The chatbot can answer, "For sure.

I just enrolled you." So that would be one goal of the chatbot.

A second example might be finding information about classes,

"Hi, what are the undergraduate level history classes offered in spring 2019?"

Then the chatbot can get back to the students and say,

"Here's the list of history classes offered in Spring 2019."

So we're making a small assumption here.

We're building a chatbot for a very restricted area.

In general, and a lot of time,

chatbots which work very well are super goal-oriented or transactional

and the state of possible utterances or requests from users is small,

smaller than what you could expect in other industrial settings.

So here we're making the assumption that the students will only

try to find information about a course,

or will try to enroll in the course.

So I want you guys to,

to pair in groups of two or three and try to come up with ideas

of what methods that we've

seen together can be used in order to implement such a chatbot.

Okay? So take a minute [NOISE] introduce yourself to your mates and,

and try to figure out which methods can be leveraged in this case.

Okay. Let's see what we have here.

RNNs for natural language processing, yeah, transfer learning.

LSTM to pick out important words from inputs based on those input triggers,

output, some predefined information from storage.

Yeah. So this seems to,

to say that there's going to be one learning part where we need to have

probably recurrent neural networks helping out and

one other knowledge base or storage part where we can retrieve some information.

We're going to see that. Some attention models is - showed that today a lot of,

uh, natural language processing models are built with attention models.

[NOISE] RNN for speech recognition and speech generation for.

So we didn't talk about the speech part.

So far we assumed that the conversational assistant is text-based,

uh, but later on we will see what happens if we wanna add speech to it.

Fancy methods.

Oh, reinforcement learning for making decisions about responses.

That's interesting. So why do you guys think we,

we would need reinforcement learning? Yes.

It allows you to have certain [inaudible] context in different states and we'll also have like a value associated with it, you said it's very goal-oriented. [inaudible]

Yeah, that's good.

So just to repeat,

it's important to keep a notion of context and also we have

a sequence of utterances from the user and, uh,

the conversational assistant and

probably the outcome of the conversation would come

far along the way and not at every step.

So that's true. Uh, Reinforcement learning has been, uh,

a research topic for conversational assistants

as well and oftentimes we will try to learn

a policy for the chatbot which given a state will tell us what action to take next.

This can be done using Q-Learning which is the method we've seen together,

or sometimes with policy gradients.

Okay. Word encoding,

sorry word embedding probably.

Okay. Cool. So I agree there's many ways to,

to plug in a deep learning algorithm in,

in this chatbot setting,

we're going to see a few of them.

First, I'd like to introduce some vocabulary which is

commonly used when talking about conversational assistants, conversational assistants.

An utterance is, you can think of it as a user input.

So if I say the student utterance,

it's the sentence that was written by the students for the chatbot.

The assistant utterance is the one coming from the chatbot side.

The intent denotes the intention of the user.

So in our case,

we will have two intents,

which is very limited.

The user either wants to find information from,

for a course or the user wants to,

uh, enroll in a class.

These are two different intentions that

are probably to be detected early on in the conversation.

And then you have something called slots.

Slots, uh, are used to gather

multiple information from the user on a specific intent that a user has.

So let's say the student wants to enroll in a class.

In order to enroll the student in a class,

you need to fill in several slots.

You need to understand probably which class the student is talking about,

which quarter the student wants to enroll in the class [NOISE] which year is the student

talking about and eventually you want to know the SUid of the students.

But probably we can assume that the SUid is already encoded

in a conversation on the environment we're in.

So these are three vocabulary and we're also going

to talk about turns for conversational assistance.

So, so single turn, uh,

conversation is when there is just a user utterance and

a response and multi-turn is when there is several user utterances,

hence, uh, conversational assistant utterances.

And you understand that

multi - multi-utterance conversations are

harder to understand because we need to track context.

Our assumption today will be that we work in

an environment with limited intents and slots.

It means we can define two intents and for each of these two intents,

there are several slots that we wanna fill in,

it's going to make our life easier.

Of course in practice you can have multi - myriads of intents and slots and you - the,

the task becomes more complicated when you have more of those.

So my first question will be,

how to detect the intent based on,

uh, the user utterance.

Can you talk about what kind of data set you need

to build in order to train the model to detect the intent?

[NOISE]

[NOISE] Or what type of network you wanna use.

[NOISE]

There's not a single good answer,

so go for it if you wanna brainstorm [NOISE].

So, I, I, I think there's,

there's going to be two options obviously because we have a,

we have a sequence coming in,

which is the user input.

We might want to use a recurrent neural network [NOISE] to encode

long-term dependencies or we might want to use a convolutional network.

Actually, uh, convolutional networks have

some benefits that recurrent neural networks don't have and they,

they might work better for example,

if the intent we're looking for is always encoded in

a small number of words somewhere in the input sequence,

because you will have a filter scanning that and the filter can detect the intent.

So, if you have a filter that was trained in order to detect the intent,

inform, another filter trained to detect the intent enroll,

then these two filters will detect the word enroll or the word I'm looking for,

and so on, in order to detect the intent, okay?

In terms of data,

what you probably need is pairs of user utterances along with the intent of the user.

So, you will need to label the datasets like this one with X and input,

I want to, so it's padded,

I want to enroll in CS106A for Winter 2019 to learn coding.

And this, you will label it as enroll.

And notice that enroll here is a function.

[NOISE] So the, the label is [NOISE] actually noted as a function.

And the reason is because we can call this function in order to retrieve information.

And another example is, hi,

what are the undergraduate level history classes offered in Spring 2019?

And this will be labeled as inform.

So it's probably a two-class classification or three classes if you

wanna add a third class that corresponds to other intent.

A user might want to use this chatbot

for another intent that the chatbot wasn't built for.

So, these are the classes, enroll and inform.

And what's interesting is that if we identify that the intent of the user is enroll,

we probably want to call an API or to request information from another server.

And in this case, it might be Axess because

the platform we use to enroll in classes is Axess.

And then, to retrieve information in order to help the user about their classes,

we can probably call Explorecourses,

assuming that these, these services have APIs,

these services have APIs.

Does that make sense? And now,

the interesting part is that the enroll function might

request some input that you have to identify.

Those will be the slots,

same for the inform function, okay?

So, we could train a sequence classifier,

either convolutional or record.

And this, we're not going to go into the details,

you've learned it in the sequence models class.

How to detect the slots now.

So in terms of data,

it's going to look very similar to the previous one,

but we will have a sequence to sequence problem now where the user utterance will be,

uh, a sequence of words and the slots tag will also be a sequence.

So, for example, show me the Tuesday

the 5th of December flights from Paris to Kuala Lumpur,

if you were to build conversational assistants for flights booking,

then the label you wanna have is probably something like that.

It doesn't have to be exactly this,

but y denotes zero.

For some of the words,

the sequences B-DAY I-DAY O O B-DEP B-ARR I-ARR.

What do you think these correspond to and why do we need that?

[NOISE] We've probably, you've probably seen that in,

in the sections a few weeks back.

[NOISE] So,

why do we denote these labels in a certain format? [NOISE]. Uh-huh.

It helps you identify the slot tag.

And like, one of them is departure, arrival,

arrival, and then the other one,

for day and possible [inaudible] [NOISE]

Yeah. Yeah, correct.

So, [NOISE] uh,

I agree with what you said for day,

day, departure, arrival, arrival.

So, these words are encoding day, departure, and arrival.

How about the B, and the I, and the O?

Yeah, someone has an idea? Yeah.

Beginning, beginning the [inaudible] one word.

Yeah, exactly. B, B denotes beginning while I denotes in or inside,

and O out or output, general.

So, what happens here is that sometimes you would have

a slot which might be filled by several words and not a single word,

and you wanna be able to detect this entire chunk.

It's called chunking. Uh, so you would use a special encoding in order to

identify if this word is the beginning of a word

that you wanna fill in the slots or is the end,

or inside, or out of the word you wanna fill in the slot.

And then, day, departure,

and arrival are three possible slots that we

wanna fill in in order to be able to book the flight.

If you don't receive these slots,

you might wanna have your chatbot request the slots later.

Okay. So, another example in, uh,

and classes here can be day, departure,

arrival, class, like, do you want to travel in eco or business,

uh, number of passengers that you wanna have on your flight.

Uh, if we were,

uh, for our chatbot here, it would be, hi,

I want to enroll in CS106A for Winter 2019 to learn coding,

and we will encode it by B,

beginning of the code of the class,

beginning of the quarter,

and beginning of the year.

[NOISE] That would be a possible encoding.

And then you will train, uh, using, ah,

probably a recurrent neural network,

an algorithm to predict all these tags. That makes sense?

So, now we have already two models that are running our chatbots.

One that is for the intent and one that is for the tags.

What do you think about joint training?

Do you think it's something we could do here?

And what do I mean by joint training?

[NOISE]

Yeah.

Training out all of the different codes,

like training for in fact quarter year and class,

rather than training in separate networks for each of those, like the joint element of the training.

Not training for different codes, no.

I was talking more about training for different tasks.

So, intent and intent for enrolling,

intent for, uh, intent and,

and uh, and slots tagging.

[NOISE] Because here we have

one intent classifier which takes an input sequence and outputs a single class,

and we have a slot tagger which takes the same input,

exactly the same input and tags every single word in the sequence.

[NOISE] So, probably we can use joint training in order to train

one network that might be able to do both.

And this network would be jointly trained with two different loss functions.

One for the intent and one for the slots.

It's usually helpful to jointly train

two networks especially in

the earlier layers because you end up learning the same type of feature.

[NOISE] That's, that's interesting for natural language processing. Yeah.

How do you do the joint loss function for it?

Does it calculate both losses independently and sum them together or is

there a trade off between training networks versus training thoughts.

So the question is how would you describe the loss function in this joint training?

You would actually sum two loss functions.

The two loss functions you were using you would just sum them and hope that's,

uh, the back propagation will train actually both networks.

And the networks will probably have a common base and then would be separated after.

So let's say you have a first LSTM layer that encodes some information about,

uh, your user utterance,

then this will give, uh,

will give its output to two different networks,

which wil - will be trained separately, okay?

And class - these here are codes for the class, quarter,

year, and SUID, assuming SUID is already in the environment,

we will not need to request it.

So can you tell me how to acquire this data now that we've seen it?

So take - take about a minute to discuss with your mates, uh,

how to acquire that type of data and then answer on Menti.

[NOISE] Okay.

So let's go over some of the answers.

[MUSIC] Mechanical Turk, have people manually collect,

annotate the data. That's true.

So as we discussed earlier in the quarter,

this would be the method which is probably the more rigorous, uh,

when it's applied with a specific, uh,

labeling process and data collection process.

It will take more time.

So you would have to build,

uh, a UI, uh,

user interface for them to be able to label all this data,

which is not trivial in general.

Amazon Mechanical Turk, pay large number of Stanford students, that works.

Have a human chat assistance service user and enter the data in, hand label data.

Yeah, I think you can start with hand labeling probably.

Can auto-generate some data by substituting date,

courses, quarter and other tags.

Oh, that's a good idea. So who wrote that?

Someone wants to comment, yeah.

I think you can already have the annotations having something like

a base at the end of the annotation [inaudible].

Yeah, that's a good idea. So I repeat for the SCPD students.

Um, we already have a bunch of possible dates.

We can easily find a list of dates.

You've done it in one assignment, right?

Uh, where you were using neural machine translation to

transfer for human-readable dates to machine-readable dates.

So we have datasets of dates,

so we could use that.

Uh, we also have a list of courses that we can probably find on ExploreCourses.

We know that there are not too many quarters and - and we ave - have

probably it's a basis for any other tag like list of

possible SUIDs or - or like seven figures, something like that.

So all numbers of seven figures hopefully.

Um, and then we can have sentences with like blank spots where we insert

these and we can generate a lot of data using this insertion scheme automated,

and every time we insert, we can label.

We're going to see that. Um, I like this idea as well.

Use a part of speech tagger,

identity recognition model to identify examples requests that are found elsewhere.

So one thing we discussed in section is

that you have available models to do part of speech tagging.

Right. So why don't we use them?

These are trained really well.

And we could give our user utterances that we collected online,

uh, and tag them automatically using these good models.

Of course, it's not gonna be perfect,

but we can at least get started with that and leverage

a model that someone else has built to tag and label our dataset.

Okay. Good ideas here.

So let's see the data generation process

which is the most strategic to start with I would say.

Uh, we would have,

talking about the - the flight booking virtual assistant.

We would have a database of all the departure locations.

So whatever, uh, Paris,

London, uh, Kuala Lumpur and a lot of arrivals as well.

So these are a list of cities that have airports probably in the world.

And we will have a list of way to write days and also class, business,

eco, eco plus, premium,

I don't know, whatever you want,

uh, and user utterances.

And then what we will do is that,

we will pull a user utterance from the database such as this one.

I would like to book a flight from dep to arrival for,

uh, in - in - in business class,

let's say in class for this day.

And then we can plug-in from dataset randomly the slots.

Does that makes sense?

We can generate a lot of data using this process.

So this user utterance can be augmented in

virtually tens or hundreds of different combinations.

So that's one way to augment your dataset automatically and label it,

but you also need hand labeled data because, uh,

you don't want your model to overfit to this specific type of user utterances, okay?

And so - so same for our virtual assistant for the - for - for the university.

Hi, I want to enroll in code for

a quarter year and then we can insert from the database the quarter,

the year and the code of different classes,

so that we can train our network on that.

Does this date augmentation makes sense?

So these are common tricks you would see in - in

various papers and this is an example of one of them.

Okay. So we can label automatically when inserting and we can

train a sequence-to-sequence model in order to fill in the slots.

Okay. So let's go on Menti and start the competition,

which is the - the most fun.

Okay. So let's get back to - to - to where we were.

We have a chatbot that is able to answer,

for sure, I just enrolled you.

The way it does that is that,

it receives the user utterance.

I want to enroll in CS106A,

Winter 2019 to learn coding.

It identifies the intent of the user using a sequence classifier,

same type of network as you've built for the Emojify assignment.

And then it also runs another algorithm,

which will fill in the slots and here we have all the slots needed.

We have the code for the class,

we have the quarter and we have the year,

the student ID is implicitly given.

So we're able to enroll - to enroll the students by calling access with all the slots.

Done. Now let's make it a little more complicated.

Let's say the students say, hi,

I want to enroll in CS106A to - to learn coding.

So the difference between this utterance and the previous one,

example one, is that you don't have all the slots.

You identify, uh, with your slot's tagger that CS106A is the code of a class,

but you don't know the quarter, you don't know the year.

So you probably want your chatbot to get back to the - to the student and say,

for which quarter would you like to enroll, right?

And the, the student would hopefully say,

Winter 2019 or Winter,

and then you have to ask for the year 2019.

And finally you can say, "For sure, I just enrolled you."

So we're not making any assumption here on natural language generation.

You've worked on a Shakespeare assignment where you generate Shakespeare-like sentences.

In fact, a good chatbot would have this feature of generating language.

But for our purpose which you can just hard-code

that when you're able to enroll the students,

you just say, I just enrolled you.

When you are able to retrieve information from the student, you would just write,

here is some information and you would plug in whatever

the ExploreCourses API sent back in a JSON, okay?

So here the idea is,

this student utterance cannot be understood without context.

There is no way to understand Winter 2019 if you don't have a context management system.

Does that make sense? So we want to build that context management system.

And then the question is how to handle context.

So there is a - there is many - there are many ways to do that,

and people are still searching for the best ways.

One way is to handle it with reinforcement learning,

as we mentioned earlier.

Another way which is quite intuitive and,

and closer to what we've seen together in sequence model,

uh, in the module,

in the module five is,

uh, this type of architecture,

which is - which is taken from Chen et al.,

End-to-End Memory Networks with

Knowledge Carryover for Multi-turn Spoken Language Understanding.

So now you're able to understand what multi-turn means and end-to-end memory networks.

So what happens here, just to describe it,

is we will save all the history utterances.

It means from the beginning of the conversation,

we will record all the utterances and messages

exchanged between the user and the, the assistant.

We will keep it in a storage that we will call history utterances.

C is the current utterance.

So let's say the student says Winter 2019.

This is the utterance of the,

the student at this point.

We will run this C,

uh, and of course like it's,

it's - this utterance will be run into

an RNN and we will get back an encoding of the sentence.

So there is all the, like,

word embedding stuff that I don't describe but you guys are used to it.

So we use word embeddings we run into, uh,

we run it to an RNN and we get back the encoding of

the user utterance and this encoding will then be compared to what we have in memory.

So all the user utterances that we had in memory,

are also going to be running in an RNN,

that will encode their information in vectors.

These vectors are going to be put in a memory representation and

RU will be directly inner product,

we, we will have an inner product from RU with all the memories and

this pooled into a softmax will give us a vector of attention,

that you guys should be used to now.

A knowledge attention distribution,

telling us what's the relation,

where should we put our attention in the memory for this specific utterance.

Does that make sense? So simple inner product softmax gives

us a series of weights here, okay?

Then we get a weighted sum of

all these attention weights multiplied by the memory and it gives us

a vector that encodes the relevance of the memory regarding our current utterance.

This is then summed and run into

a simple matrix multiplication to get an output vector which

will be run in a slot stacking sequence and usually it's experimental

but they pass also the current utterance to the RNN tagger,

and the RNN tagger comes up with a slot tagging.

So using that you can understand that Winter 2019 is

actually the tagger for the slot's quarter and year,

because you have this memory network.

Does that makes sense? So this is another type of

attention models you want to use and

these memory networks can be used to manage some contexts for the slots tagger.

Okay. So just to recap,

we have our example,

"Hi I want to enroll in a class," and we detect the intent, which is enroll.

We also detect that there's some slots missing because we know,

we know that the enroll function needs the quarter,

the year and the class in order to be able to be called.

So we have to ask for those.

So we probably hard coded the fact that if you don't have the quarter,

the year and the class,

you probably want to first ask for the class,

or the quarter, or the year.

Then you can, you can get back to the person by asking,

"Which class do you want to enroll in?"

The person would get back to you,

you will use your memory network to understand that CS230 is

the slots for the enroll intent, you would fill it in.

So now we have our intent with the class equals CS230,

and we have our slots quarter and year which are to be filled,

the chatbot gets back, "For which quarter?"

and hopefully the student gives you the year at the same time and you can fill in

the slots and then you're enrolled in CS230 for Winter 2019. Yeah.

Should be Spring.

Should be Spring, yeah [LAUGHTER].

This chatbot is not trained very well.

[LAUGHTER] Okay. Any questions on that?

So this is a very simple case of a,

of a conversational assistant.

Just to give you some ideas.

There's some papers listed in the presentation that you can go to in order to

get more advanced, uh, research insights.

Uh, but the idea here is that we are limited to a specific intent,

to two specific intents and a few slots.

What do you think we would need if we didn't

restrict ourselves to specific intents and slots?

[NOISE]

Very complicated, yeah.

One industrial way to do it is to use a knowledge graph.

What it means is let's say you're an e-commerce platform,

you probably have, from your platform,

a knowledge graph of,

of all the items on the platform with connections among them,

like let's say color of let's say you have a shoe,

a shoe is a slot that might be the object for the intent I want to buy something, right?

The shoe can have several attributes like color or size or men or women,

like gender and all these are connected together in a ge - in a,

in a, in a, in a gigantic knowledge graph.

And you will follow the path of this knowledge graph

following some probability - probabilities.

So when we detect the intent of the user which is,

buy something, we could identify the object, I wanna buy a shoe,

and then based on our knowledge graph,

it says that the next question that we should

ask or the next slots that we need to fill is,

uh, which brand do you want your shoe to be?

And so the knowledge graph is going to tell you with 60 percent probability,

go to brand and ask about the brand.

Once you're there what other information you need in order to be able to

retrieve five results for the user to review, and so on.

So the knowledge graph is something industrial that can be used

in order to have multiple intents,

multiple slots for every intent.

Okay. And at the end we can make an API call here.

With CS230 quarter, Winter, 2019,

quarter, Winter, year 2019, and the SUid.

Okay. Another question I'll,

I'll had for you I - I've had for you, I,

I have for you is, uh,

how to evaluate the performance of a chatbot.

What do you think of that?

So the - there are common ways to [NOISE] to,

to evaluate several part of your pipeline like,

how is your slot tagger doing?

How is your intent classifier doing.

You can use metrics such as precision recall,

F1 score, which is a mix of both,

uh, and report those in order to compare how this module is doing for the chatbot.

But ultimately you want to understand how good is your chatbot overall.

So some experiments are done and this is

a paper of a deep reinforcement learning chatbot,

built in 2017 by the MILA, uh, Serban et al.

and what they did is that they used Mechanical Turk in order to

evaluate their chatbot and also built

a scoring system for their reinforcement learning chatbot.

So I'm reading for you the instructions.

Uh, you will be presented with a conversation between two speakers,

Speaker A and B.

You will also be presented with four potential responses from one of the speakers for

this dialogue and the task is for you to rate each of, uh,

the responses between one inappropriate,

doesn't make sense, to five, uh,

highly appropriate and interesting based on how appropriate the response is,

to continue the conversation,

three is neutral and,

uh, if two responses are equally appropriate,

you should give them the same score and if you see a response that is not in English,

please give it one score.

So here's what happens,

uh, from a user perspective.

You will have a conversation,

you need to work on your English.

Why do you say tha - that about me?

Uh, well your English is very poor.

So this is a conversation.

And then the response 1 is,

but English is my native language.

Response 2 is, what are the reasons come to mind?

Response 3 is, here is a funny fact.

Go is the shortest complete sentence in the English language and

then the fourth response is bye doggie [LAUGHTER].

So obviously you have to, you have to score, uh,

you have to score these, uh,

these responses according to what you think how relevant they are.

And then, um, and then this course will be used either for the scoring system

of the deep reinforcement learning chatbot or it can be

used to evaluate how good is your chatbot compared to other chatbots.

So maybe each of these responses come from a different model. Does that makes sense?

So these are a few ways.

Then another way, uh,

which is asking for the opinion of the user on,

uh, different, uh, responses.

So let's say, you - you're a user and,

um, you're - you are comparing two chatbots.

You can give your opinion on which one you think is more natural.

And you would ask a lot of users to do that to compare

two or three chatbots together and also compare them to natural language from human.

And then by doing a lot of,

uh, Mean Opinion Score, uh,

experiments, you can evaluate which chatbots are better than the others,

just comparing them one to one.

Okay. Now getting back

to one thing that a student

mentioned earlier is what if we want to have a vocal assistant?

So right now our assistant is not vocal, it's just text.

What other things do we need to build in order to make it a vocal assistant?

We're not going to go into the details but roughly you

would need a speech-to-text system,

which will take the voice of a user,

convert it into a text,

and this as you've seen in the sequence model class,

has different step in the pipeline uh, and the speech-to-text.

So - and text-to-speech.

That takes the text from the chatbot and converts it into a voice.

So that's how you have like virtual assistants talking to us,

it's because they have a text-to-speech system running.

And these are three papers,

the first one is Deep Speech 2 from Baidu's team, uh,

which built an end-to-end speech recognition in English and Mandarin;

and the two others are text-to-speech synthesis.

So one came up in February 2018 which is the Tacotron 2

and the second one is WaveNet which is a very popular generative models,

and these are - these are far beyond the scope of the class uh,

but, uh you can study them in other classes

at Stanford which are more specific to speech.

Uh, okay, class project advice;

so this Friday we are going to go over uh,

again the rubrics of what we look at when we - we

create projects and here is the list of things we would look at, uh.

So make sure you have a very good problem description.

When you read papers you see that there is a very good abstract.

We expect you to give us a very good abstract so that when we read it,

we get a good understanding of the paper.

Hyperparameter tuning; always report what you do.

You don't need to - to be very exhaustive but - but you can just

tell us what hyperparameters you've been choosing

and which ones you've been testing and why they didn't work.

Um, the writing - we look for typos.

This is common in - in the grading scheme.

Typos, uh, uh, clear language.

So review it, peer review your paper.

Explanation of choice and decision;

this is a very important part.

We expect you to explain uh,

the decisions you're making.

So we don't want you to tell us,

"I've taken - I've made that decision" just without explaining.

But rather tell us,

"there is this paper that mentioned that,

this architecture worked well on that specific task.

Uh I've tried three architectures here are my hyperparameters and results.

That's why I'm going to - I'm going to dig more into that one" and so on.

Data cleaning and preprocessing,

if applicable to your pro - project, explain it, uh,

how much code you wrote on your own; it's important to us,

and please submit your GitHub privately to the TAs.

When you submit your project, it's going to make it easier for us to review the code.

Um, insights and discussions include; the next steps.

What would you have done if you had more time?

Uh, and also interpret your results.

Don't just give results without explanation but

rather try to extract information from these results,

and it can also drive your next steps explanation.

Uh, results are important but,

if you don't have the results you expected, it's fine.

We will look at how much work you've done and some tasks are very complicated.

We don't expect you to beat state of the art on every single task.

Some of you are going to beat state of the art hopefully,

but those of you who didn't;

still report all your results and explain why it didn't work.

Uh, give references and also penalty for more than five pages.

So if you're working on a - on

a theoretical project you can add additional pages as appendix.

You can also add appendix for your project.

But the core has to be five pages.

And for the final poster presentation which will happen not this Friday;

next one, uh, we will ask you to pitch your project in three minutes.

So not everyone in the group has to talk but at least one person has to talk,

an - and we prefer if several of you talk in the project,

but you have three minutes to pitch your projects.

So prepare a pitch in advance and you will have two minutes of questions from the TA,

which are also part of the grading.

Okay? Finally, what's next after CS230?

So there's a ton of classes at Stanford.

We're in a good learning environment,

which is - which is super.

Uh, next steps can be in the university classes,

you can take natural language processing and, uh,

computer vision but also [NOISE] classes from different departments.

Uh, deep generative models is a good way to learn about text-to-speech.

For example or GANs.

Probabilistic graphical models is also a very important class in the CS department.

Of course if you haven't taken it yet CS 229; machine learning,

or CS 229A; applied machine learning,

are the go to - to learn machine learning.

Reinforcement learning is a class where you can - you can delve more into Q-learning,

policy gradients, and all these methods uh,

that sometime use deep learning.

So we're going to publish that list in case you want to check it.

But these are examples of classes you can take and of

course there are other classes that are not mentioned here,

that might be relevant to

pursue your learning in deep - deep learning and machine learning.

Okay. That said, I'm going to - to give the microphone to Andrew.

For closing remarks and - and yeah good luck on your projects.

So we'll see you on Friday for

the discussion sections and next week for the final project. Do you have a microphone?

Thanks, Kian. So, all right. Here we are,

at the end of this class.

Nearly at the end this class.

Um, you know, the NeurIPS conference is taking place right now.

Formerly the NIPS Conference, but they renamed to NeurIPS,

and I remember it was ten years ago that um - at that time a PhD student [inaudible]

presented the paper; workshop paper at NIPS telling people, "hey,

consider using GPUs, and CUDA there which is a new thing

that NVidia had just published to train neural networks."

We've done that work on a GPU server,

that Ian Goodfellow

the creator of GANs had built in his dorm room when he was an undergraduate at Stanford.

So our first GPU server was built in a Stanford undergrad's dorm room.

Um, and I remember sitting down with Jeff Hinton at the food court and saying,

"hey check out this CUDA thing" and Jeff say,

"oh, but GPU programming is really hard".

But then - but then, but oh,

maybe this CUDA thing looks promising.

Um, and I tell this story because I want you to

know as Stanford students that your work can matter, right?

When Ian Goodfellow built that GPU server in his dorm room,

I had no idea if he realized that a decade later you

know someone would be winning several hundred hours of AWS credits

to train bigger deep learning algorithms.

But I think at Stanford - here at Stanford University,

we're very much at the heart of the technology world.

I think Silicon Valley is here to a large part

because Stanford University is here, and um,

we live in a world where with the superpowers that you now have,

you have a lot of opportunities to do new and exciting work.

Which may or may not seem like it will matter in the short run.

Maybe even seem consequential in the short run.

Because certainly you have a huge impact in the long run.

Actually, a couple of weekends ago,

so my wife - we roast coffee beans at home, right?

My wife buys raw coffee beans and then we actually roast them, and Carol;

my wife, tends to roast them in this really cheap popcorn popper that we have right now.

So I don't have - I don't know how much coffee you guys drink, I drink a lot of coffee.

And so you know - so Carol buys these green coffee beans,

she puts them in this cheap popcorn popper;

which is made for popping popcorn,

not made for roasting coffee beans.

This is one of the standard cheap ways to roast coffee beans and I love my wife.

I drink the coffee she makes but sometimes she burns the coffee beans.

So I - I found this article on the Internet from a former student

that had written an article on how they use machine learning to

roast - to - to optimize the roasting of coffee beans,

um, and so I forwarded it to Carol, um.

She wasn't very happy about that. [LAUGHTER], um.

And I raised this as another [LAUGHTER] example of how, uh, uh,

all of you, um - you know,

I would never have thought of applying machine learning to roasting coffee beans.

It's just - I mean, you know,

I like my coffee but it had never occurred to me to do that.

But someone taking a machine learning class,

like you guys are,

go ahead and come up with a better way of

roasting coffee beans using learning algorithms.

Um, and again, I think - I don't know if

this particular person who wrote this blogpost was thinking of building a business out of it.

I, I don't know. There might be a business there,

there might not, or it might be just a fun personal hobby. I actually don't know.

Um, but all of you with these skills have that opportunity.

And then, um, again,

earlier this week, uh,

was it Monday night?

Um, a group of us, uh, we,

we were actually in the Gates Building, um,

where a bunch of students actually from the AI for Healthcare Bootcamp that,

that Kian alluded to,

you know, we're going over some of the final projects for the,

for the students in the AI for Healthcare Bootcamp,

um, were, were working on.

And I think, and I think I actually met several people including, Aarti right?

When she first participated in a much earlier version of that AI for Heathcare Bootcamp.

So you, you can ask Aarti about it if you're interested.

But they're - um, one of the, um,

master's students who was working with Ph.D student

Pranav Rajpurkar that I think you guys met in this class.

He was demoing an app where, um,

you could pull up an X-ray film,

uh, and take a picture with your cell phone,

um, upload the picture to a website, um,

and have a website read the X-ray and suggest a diagnosis for a, for a patient.

Um, most of the planet today has insufficient access to radiology services.

Uh, there are many countries where it costs you three months of salary,

um, to go and get an X-ray taken,

and then maybe try to find a radiologist to read it.

But most of the planet,

um - billions of people on this planet do

not have sufficient services - radiology services.

And, um, while the Stanford students in the AI for

Healthcare Bootcamp is doing a research project - Actually,

you were a co-author on the CheXNet paper, weren't you, Aarti? Yeah, right.

Yes. Aarti was a shared co-author on,

on, one of these papers.

Um, it is again maybe work done here at Stanford that, you know,

is taking the first steps toward maybe - if we can improve the deep learning algorithms,

pass regulatory hurdles, you know,

prove safety, maybe that type of work is happening here at Stanford.

Doing AI for Healthcare, maybe that will have

a transformative effect on how healthcare is run,

um, all around the world.

So, um, the skills that you guys now have,

uh, uh, are a very unique set of skills.

There are not that many people on the planet today that can

apply learning algorithms and deep learning algorithms the way that you can.

And you can tell a lot of the ideas you learn in this class were,

you know, invented in the last year or two.

So there's just not yet been time for these ideas to even become widespread.

And if I look at a lot of the most pressing problems facing society,

be it a lack of access to healthcare or,

um - Scientists spend a lot of time thinking about climate change.

Um, uh, and I think if you look at the,

the, the - can we improve access to education.

Can we just make whole society run more efficiently.

Um, I think that all of you have the skills to do very unique projects.

Um, and I hope that as you graduate from this class,

I'm sure some of you will build great businesses,

maybe make a lot of money. That's great.

And, and I hope that all of you will also take the unique skills you

have to work on projects that matter the most to other people,

that, that help other people.

Um, because if one of you does not take your skills to do something meaningful,

then there's probably some very meaningful project that just no one is working on.

Because I think the number of meaningful projects, um,

I think actually greatly exceeds the number of

people in the world today that are skilled at deep learning.

Which is why all of you have

a unique opportunity to take these algorithms that you now know about,

to apply to anything from, I don't know,

developing novel chatbots, to improving healthcare,

to - I guess my team are learning AIs.

Improving manufacturing, agriculture, also some healthcare,

to maybe helping with climate change,

to helping with global education, uh, and,

and any other problems that, that really matter.

So I hope, I hope maybe - uh,

I hope that all of you go on,

um, to, to do work that matters.

Um, and then one last story, um, you know,

a few - jus - just a few months ago now,

I got to drive a tractor, right?

It was very big, a little bit scary.

It feels like a bigger machine than I should be qualified to drive.

Um, it's, it's a huge tractor.

And, and it turns out that when you drive

a tractor - so it turns out when you drive a normal car,

you know, it's really clear which way is up on your steering wheel, right?

You point the steering wheel up and your,

your car drives forward.

Uh, for the tractor that I got to drive,

this huge tractor, it turns out that, uh,

this giant steering wheel and to drive straight,

the giant steering wheel was just oriented at a weird angle.

And to turn right,

you turn it clockwise.

To turn left, you turn it anticlockwise,

and, and that was that,

right? So it was a lot of fun.

Um, and maybe in addition to,

uh - and, and it was just fun, you know [LAUGHTER].

I drove a tractor, made a u-turn,

drove back to where I started,

did not hit anyone,

you know, there was no accident.

And then I climbed down off this giant tractor.

Um, and maybe I tell that story because, uh, uh,

I hope that even while you are doing this important, uh, uh,

uh, maybe beneficial to other people, these types of work,

I hope, I hope you also have fun.

I, I think that I feel really privileged that as a Machine Learning Engineer,

um, I some days,

I get to go drive a tractor, right?

And, and I hope that, um - and one of the most exciting things,

um, you know, I, I, I feel like,

um, uh, a lot of the best - a lot of biggest untapped opportunities for AI,

lie outside the software industry.

Um, I'm very proud of the work that helped to,

you know, leading the Google Brain team, leading AI Baidu.

And I think more people should do that type of work.

Um, and I think that,

um, here in Silicon Valley,

many of you will get jobs in the tech sector, and that's great.

We need more people to do that.

And I also think that if you look at all of human activity,

the majority of human activity is actually outside the software industry.

The majority of global GDP growth,

global GDP, uh, uh, is actually outside the software industry.

And I would just urge you,

as you're considering what is the most meaningful work,

to consider software industry,

but also look outside the software industry.

Because I think really the biggest untapped opportunities for

AI lie outside - I think lie outside the software industry.

And, um, and we can't have everyone doing the same thing, right?

That's actually not a healthy plan,

and if everyone, you know,

works on improve web search or improve - or,

or, or even improve healthcare.

I, I - and I think we need a world where all of you have these skills,

share these skills, teach other people what you've learned,

and go out to do this work that hopefully affects the software industry,

affects other industries, affects for-profit, non-profit, affects government.

Um, but use these AI capabilities to lift up the whole human race, right?

Um, and then, finally, um, uh,

the last thing I want to say on behalf of Kian and me,

and the whole teaching team, is, um,

I wanted to thank you for your hard work on this class.

Uh, I know that, you know,

watching the videos, uh,

doing the homeworks on the website, uh,

meeting with the TAs, uh,

going to the section, uh,

um, you know, that many of you have put a lot of work in this class.

And it wasn't so long ago,

I guess, when I was a student.

Um, you know, staying at home,

doing this homework or trying to derive that math thing.

I, I also take some online classes myself,

so it's actually not so long ago that, you know,

I was sitting at a computer much like you trying to watch some Coursera videos,

and then click on this, click on that,

and answer things online.

Uh, and, and, I, I, I appreciate.

Uh Kian and I, and the whole teaching team appreciate all the hard work you've put into this.

Um, and I hope also that you got a lot out of

your hard work and that you will take these rare and unique skills you now have to go on.

And, and when you graduate from Stanford - or for the - or for the home viewers,

I guess, uh, for the - uh, for the home viewers,

as well as the, uh, in-classroom viewers.

That you take these rare skills that you now have and go on to do work that matters,

and go on to do work that helps other people.

So with that, um, I look forward to seeing,

um, all of your projects at the poster session.

Uh, uh, and I apologize in advance,

we won't be able to really get a deep understanding in three minutes, but don't worry.

We do read your project reports.

Uh, but I look forward to seeing - uh,

I hope you're looking forward also to seeing everyone else's work at the poster session.

Uh, but with that, let me just say on behalf of the - Kian and me,

and the whole teaching team, uh,

thank you all very much.

Thank you. [APPLAUSE]


知识点

重点词汇
embedding [ɪm'bedɪŋ] v. [医] 植入；埋藏（embed的ing形式） { :6007}

denotes [diˈnəuts] 为...的符号，表示，指示（denote的第三人称单数） { :6148}

denote [dɪˈnəʊt] vt. 表示，指示 {cet6 ky toefl ielts gre :6148}

dependencies [dɪ'pendənsɪz] n. 依赖性，相关性；管理 { :6214}

leveraged [ˈli:vəridʒd] adj. 杠杆的；杠杆作用的 v. （美）杠杆式投机，举债经营（leverage的过去分词形式） { :6493}

leverage [ˈli:vərɪdʒ] n. 手段，影响力；杠杆作用；杠杆效率 v. 利用；举债经营 {cet4 :6493}

randomly ['rændəmlɪ] adv. 随便地，任意地；无目的地，胡乱地；未加计划地 { :6507}

oriented ['ɔ:rɪəntɪd] adj. 导向的；定向的；以…为方向的 v. 调整；使朝向（orient的过去分词）；确定…的方位 { :6514}

tractor [ˈtræktə(r)] n. 拖拉机；牵引机 {gk cet4 cet6 ky toefl ielts :6548}

rigorous [ˈrɪgərəs] adj. 严格的，严厉的；严密的；严酷的 {cet6 ky toefl ielts :6606}

hurdles [ˈhə:dlz] n. 障碍；跨栏；跨栏跑；障碍赛跑（hurdle的复数） { :6611}

jeff [dʒef] n. 杰夫（男子名，等于Jeffrey） { :6731}

inaudible [ɪnˈɔ:dəbl] adj. 听不见的；不可闻的 { :6808}

algorithm [ˈælgərɪðəm] n. [计][数] 算法，运算法则 { :6819}

algorithms [ˈælɡəriðəmz] n. [计][数] 算法；算法式（algorithm的复数） { :6819}

myriads ['mɪrɪədz] n. 无数，极大数量( myriad的名词复数 ) { :7106}

Turk [tә:k] n. 土耳其马；土耳其人 { :7202}

recurrent [rɪˈkʌrənt] adj. 复发的；周期性的，经常发生的 {ky :7228}

gradients [ˈgreɪdi:ənts] n. 渐变，[数][物] 梯度（gradient复数形式） { :7370}

graphical [ˈgræfɪkl] adj. 图解的；绘画的；生动的 { :7466}

reinforcement [ˌri:ɪnˈfɔ:smənt] n. 加固；增援；援军；加强 { :7506}

phd [ ] abbr. 博士学位；哲学博士学位（Doctor of Philosophy） {ielts :7607}

roasting [ˈrəʊstɪŋ] n. 烧烤 v. 烤，烘；煅烧（roast的ing形式）；使变暖 adj. 烤肉用的；灼热的 adv. 炙热地 { :7679}

roast [rəʊst] n. 烤肉；烘烤 adj. 烘烤的；烤过的 vt. 烤，焙；烘，烘烤；暴露于某种热力下以得温暖 vi. 烤；烘 {gk cet4 cet6 ky toefl ielts :7679}

gigantic [dʒaɪˈgæntɪk] adj. 巨大的，庞大的 {cet6 ky toefl ielts :7783}

Et ['i:ti:] conj. （拉丁语）和（等于and） { :7820}

automated ['ɔ:təʊmeɪtɪd] adj. 自动化的；机械化的 v. 自动化（automate的过去分词）；自动操作 {toefl :8095}

silicon [ˈsɪlɪkən] n. [化学] 硅；硅元素 {cet6 ky :8279}

superpowers [ˈsju:pəˌpaʊəz] n. 超级大国( superpower的名词复数 ) { :8289}

encode [ɪnˈkəʊd] vt. （将文字材料）译成密码；编码，编制成计算机语言 { :8299}

encodes [enˈkəʊdz] v. （将文字材料）译成密码( encode的第三人称单数 ); 编码，编制成计算机语言 { :8299}

encoded [ɪn'kɒdɪd] adj. [计] 编码的 v. 把…编码（encode的过去分词） { :8299}

encoding [ɪn'kəʊdɪŋ] n. [计] 编码 v. [计] 编码（encode的ing形式） { :8299}

dorm [dɔ:m] n. 宿舍（等于dormitory） n. (Dorm)人名；(瑞典)多尔姆 {cet4 cet6 ky :8314}

tha [,ti ɛtʃ 'e] abbr. thaumatin 竹芋蛋白 { :8395}

enrolled [en'rəʊld] v. 登记（入会；卷） { :8506}

enrolling [en'rəʊlɪŋ] n. 入学；登记 v. 注册；参加；招收（enroll和enrol的ing形式） { :8506}

augment [ɔ:gˈment] n. 增加；增大 vt. 增加；增大 vi. 增加；增大 {cet6 ky toefl ielts gre :8589}

augmented [ɔ:g'mentɪd] adj. 增广的；增音的；扩张的 { :8589}

alluded [əˈlu:did] vi. 暗指，转弯抹角地说到；略为提及，顺便提到 { :8601}

renamed [ˌri:ˈneɪmd] v. 给…重新取名，改名( rename的过去式和过去分词 ) { :8668}

intuitive [ɪnˈtju:ɪtɪv] adj. 直觉的；凭直觉获知的 {gre :8759}

implicitly [ɪm'plɪsɪtlɪ] adv. 含蓄地；暗中地 { :8775}

quiz [kwɪz] n. 考查；恶作剧；课堂测验 vt. 挖苦；张望；对…进行测验 {gk cet4 cet6 ky :8784}

popcorn [ˈpɒpkɔ:n] n. 爆米花，爆玉米花 {gk toefl :8839}

insertion [ɪnˈsɜ:ʃn] n. 插入；嵌入；插入物 { :9116}

neural [ˈnjʊərəl] adj. 神经的；神经系统的；背的；神经中枢的 n. (Neural)人名；(捷)诺伊拉尔 { :9310}

metrics ['metrɪks] n. 度量；作诗法；韵律学 { :10163}

conversational [ˌkɒnvəˈseɪʃənl] adj. 对话的；健谈的 {toefl :10168}

delve [delv] n. 穴；洞 vi. 钻研；探究；挖 vt. 钻研；探究；挖 n. (Delve)人名；(英)德尔夫 {gre :11237}

academia [ˌækəˈdi:miə] n. 学术界；学术生涯 { :11277}

clockwise [ˈklɒkwaɪz] adj. 顺时针方向的 adv. 顺时针方向地 {cet6 ky toefl ielts :11482}

exhaustive [ɪgˈzɔ:stɪv] adj. 详尽的；彻底的；消耗的 {toefl gre :11495}

optimize [ˈɒptɪmaɪz] vt. 使最优化，使完善 vi. 优化；持乐观态度 {ky :11612}

mandarin [ˈmændərɪn] n. （中国）普通话；国语；官话；满清官吏；柑橘 adj. 紧身马褂的 { :11837}

manually ['mænjʊəlɪ] adv. 手动地；用手 {toefl :12167}

Demoing [ ] (Demo 的现在分词) n. 演示, 样本唱片 [计] 演示 { :12334}

amazon ['æməzən] 亚马逊；古希腊女战士 { :12482}

oftentimes [ˈɒfntaɪmz] adv. 时常地 { :12676}

propagation [ˌprɒpə'ɡeɪʃn] n. 传播；繁殖；增殖 {cet6 gre :12741}

multiplication [ˌmʌltɪplɪˈkeɪʃn] n. [数] 乘法；增加 {cet6 :12748}

healthcare ['helθkeə] n. 医疗保健；健康护理，健康服务；卫生保健 {ielts :13229}

chen [ ] n. 陈 { :13288}

rubrics ['ru:brɪks] n. （用红字印刷的）题目（或句子等）( rubric的名词复数 ); 注释; 写在礼拜规程里的礼仪指示; 习俗 { :13513}

transformative [ˌtræns'fɔ:mətɪv] adj. 变化的，变形的；有改革能力的 { :15252}

consequential [ˌkɒnsɪˈkwenʃl] adj. 间接的；结果的；重要的；随之发生的；自傲的 {gre :15298}

stanford ['stænfәd] n. 斯坦福（姓氏，男子名）；斯坦福大学（美国一所大学） { :15904}

multi ['mʌlti] pref. 多 { :16015}


难点词汇
APP [æp] abbr. 应用（Application）；穿甲试验（Armor Piercing Proof） n. (App)人名；(英)阿普 { :16510}

upload [ˌʌpˈləʊd] vt. 上传 { :16624}

eco [ ] abbr. 电子耦合振荡器（Electron-Coupled Oscillator）；环境控制组织（Environmental Control Organization）；经济合作组织（Economic Cooperation Organization） n. (Eco)人名；(意)埃科 { :17773}

radiologist [ˌreɪdiˈɒlədʒɪst] n. 放射线研究者 { :17826}

labeling ['leɪblɪŋ] n. 标签；标记；[计] 标号 v. 贴标签；分类（label的现在分词） { :17997}

untapped [ˌʌnˈtæpt] adj. 未开发的；未使用的；塞子未开的 {toefl gre :18033}

datasets [ ] (dataset 的复数) [电] 资料组 { :18096}

dataset ['deɪtəset] n. 资料组 { :18096}

jus [ʒu:] n. （拉）法律；权利；法律原则；（法）汁液 { :19069}

annotate [ˈænəteɪt] vt. 注释；作注解 vi. 注释；给…作注释或评注 {toefl gre :19833}

brainstorm [ˈbreɪnstɔ:m] n. 集思广益；头脑风暴；灵机一动 vt. 集体讨论；集思广益以寻找 vi. 集体讨论；动脑筋；出主意 { :20387}

generative [ˈdʒenərətɪv] adj. 生殖的；生产的；有生殖力的；有生产力的 { :21588}

doggie ['dɒgɪ] n. 小狗，狗 { :22080}

Hinton [ ] [人名] [英格兰人姓氏] 欣顿住所名称，来源于古英语，含义是“高，或户，修道院+圈用地，居留地”(high,or household,monastery+enclosure,settlement); [地名] [加拿大、美国、英国] 欣顿 { :22297}

transactional [træn'zækʃənəl] adj. 交易型的；事务性的；事务处理的 { :22587}

augmentation [ˌɔ:ɡmen'teɪʃn] n. 增加，增大；增加物 {gre :22669}

annotations [ ] 释文 { :22939}

annotation [ˌænə'teɪʃn] n. 注释；注解；释文 { :22939}

radiology [ˌreɪdiˈɒlədʒi] n. 放射学；放射线科；X光线学 { :23300}

recap [ˈri:kæp] n. 翻新的轮胎 vt. 翻新胎面；扼要重述 { :23344}

popper [ˈpɒpə(r)] n. 服药成瘾者；发出啪声的人或物；爆竹；炮手；爆玉米花机 n. (Popper)人名；(英)波珀；(德、匈、捷、芬、瑞典、法)波佩尔 { :23767}

kuala [ ] [地名] [印度尼西亚] 瓜拉 { :24053}

lumpur [ ] n. 吉隆坡 { :24724}

typos ['taɪpəʊs] n. 打字错误 { :25550}

GE [ʒei] abbr. 美国通用电气公司（General Electric Co.）；总能量（gross energy） n. (Ge)人名；(朝)揭；(俄)格 { :25836}

ave [ˈɑ:vi] abbr. 途径；林荫大道；大街（avenue） { :26490}

clicker ['klɪkə] n. 排字工头；鞋匠工头；站在店外招徕顾客者；遥控器 { :26745}

probabilistic [ˌprɒbəbɪˈlɪstɪk] adj. 概率性的；或然说的，盖然论的 { :27390}

embeddings [ɪm'bɛd] v. [医] 植入；埋藏（embed的ing形式） { :27523}

carryover [ˌkærɪ'əʊvər] n. 移后扣减；移行 { :27537}

predefined [pri:dɪ'faɪnd] adj. [计] 预先定义的 v. 预先确定（predefine的过去式） { :30849}

classifier [ˈklæsɪfaɪə(r)] n. [测][遥感] 分类器； { :37807}

wil [wɪl] [计]= White Indicator Lamp,白色指示灯 { :38036}

tagger ['tægə] n. 薄铁片；垂下物；尾随者；加标签者 { :38754}

Goodfellow [ ] [人名] [英格兰人姓氏] 古德费洛绰号，意气相投的伙伴，来源于中世纪英语，含义是“好+伙伴”(good+fellow) { :39174}

anticlockwise [ˌæntiˈklɒkwaɪz] adj. 逆时针的；反时针方向的 adv. 逆时针地 { :41871}


生僻词
aarti [ ] n. (Aarti)人名；(芬)阿尔蒂

baidu [ baɪ'du] n. 百度（全球最大的中文搜索引擎）

blogpost [ ] [网络] 添加新博客；消息；消息条目

bootcamp [ ] n. 训练营地

chatbot [ ] [网络] 聊天机器人；夏堡雅邑；网络聊天系统

chatbots [ ] [网络] 自动化语音聊天程式

convolutional [kɒnvə'lu:ʃənəl] adj. 卷积的；回旋的；脑回的

Coursera [ ] [网络] 免费在线大学课程；免费在线大；斯坦福

dep [ ] abbr. 或然的偏转误差（Deflection Error Probable）；紧急民防计划（Domestic Emergency Plan） n. (Dep)人名；(越)美；(法)德普

GitHub [ ] [网络] 源码托管；开源项目；控制工具

google [ ] 谷歌；谷歌搜索引擎

hyperparameter [ ] [网络] 超参数；分别有一个带有超参数

hyperparameters [ ] [网络] 超参数；超參數

kian [ ] [网络] 奇恩；奇安；吉安

nvidia [ɪn'vɪdɪə] 英伟达，一家以设计显示芯片和主板芯片组为主的半导体公司

overfit [ ] [网络] 过拟合；过度拟合；过适应

preprocessing [prep'rəʊsesɪŋ] n. 预处理；预加工

serban [ ] [网络] 戴白头巾；瑟尔邦

softmax [ ] [网络] 柔性最大传递函数；前回收的日志文件的百分比；西风狂诗曲系列篇章

suid ['su:d] [医][=sudden unexpected infant death]婴儿意外猝死

tens [ten] 十位


词组
a ton [ ] [网络] 一吨

a ton of [ ] [网络] 许多；大量的

allude to [ ] v. 暗指 [网络] 提到；暗示；提及

Amazon Mechanical Turk [ ] [网络] 亚马逊土耳其机器人；亚马逊人端运算平台；亚马逊的土耳其机器人网站

an algorithm [ ] [网络] 规则系统；运算程式

applicable to [ ] adj. 适用于；能应用 [网络] 可应用于；适用的；适应于

back propagation [ˈbækˌprɔpəˈgeɪʃən] [网络] 反向传播；误差反向传播；反向传播算法

behalf of [ ] 代表

beneficial to [ ] adj. 有益于 [网络] 对……有利；有益的；有利于

blank spot [ ] un. 空白点 [网络] 没有达到的方面或部分；板面上有几个空位

click on [klik ɔn] [网络] 点击；用鼠标点击；单击

dorm room [ ] 宿舍 寝室

enrol in [ ] 参加，(使)加入(某团体、组织等)，入会；入学；被登入名册；参军，入伍

enroll in [ ] v. 登记 [网络] 报名参加；参加；加入

et al [ ] abbr. 以及其他人，等人

et al. [ˌet ˈæl] adv. 以及其他人；表示还有别的名字省略不提 abbr. 等等（尤置于名称后，源自拉丁文 et alii/alia） [网络] 等人；某某等人；出处

et. al [ ] adv. 以及其他人；用在一个名字后面 [网络] 等；等人；等等

Fancy ! [ ] 真想不到!竟有这样的事!

giant tractor [ ] 巨型拖拉机

graphical model [ˈɡræfikəl ˈmɔdəl] n. 图模型 [网络] 图形模型；图模式；图形化模型

if applicable [ ] [网络] 如适用；如果适用；如适用者

in the pipeline [in ðə ˈpaɪpˌlaɪn] 在考虑（或规划、准备） 中; 在酿中; 在准备中; 在完成中

Knowledge Graph [ ] 知识图

Kuala Lumpur [.kɑ:lә'lumpuә] n. 【城】吉隆坡；【旅】吉隆坡 [网络] 马来西亚；吉隆玻；吉隆坡市

labeling process [ ] 《英汉医学词典》labeling process 标记过程

learning algorithm [ ] [网络] 学习演算法；学习算法；学习机制

machine translation [məˈʃi:n trænsˈleiʃən] n. 机器翻译；计算机翻译 [网络] 机骗译；机译；机器翻译技术

matrix multiplication [ ] [网络] 矩阵乘法；矩阵相乘；三阶矩阵乘法

Mechanical Turk [ ] 土耳其机器人

multiple slot [ ] 多翼缝

multiply by [ ] v. 乘 [网络] 乘以；乘上；使相乘

neural network [ˈnjuərəl ˈnetwə:k] n. 神经网络 [网络] 类神经网路；类神经网络；神经元网络

neural networks [ ] na. 【计】模拟脑神经元网络 [网络] 神经网络；类神经网路；神经网络系统

of intent [ ] 有意地,故意地,蓄意地

on ... behalf [ ] [网络] 代取件业务；代订；代为

on behalf [ ] [网络] 代取件业务；代订；代为

on behalf of [ɔn biˈhɑ:f ɔv] na. 替；为…的利益 [网络] 代表；代表某人；以…名义

on the website [ ] [网络] 在网站上

output vector [ ] 输出矢量

Ph D [ ] abbr. 〈外(=Philosophiae Doctor)哲学博士 [网络] 医学博士；博士学位；哲学博士学位

plug in [plʌɡ in] v. 插入；插插头；插上 [网络] 插件；连接；插上电源

popcorn popper [ ] [网络] 爆米花机；爆米花波普尔；国际爆米花协会添购新兵器

poster session [ ] 图片式讨论会；海报论文

recurrent neural network [rɪˈkɜ:rənt ˈnjuərəl ˈnetwə:k] [计]递归神经网络

reinforcement learning [ ] 强化学习

roast coffee [ ] 焙炒咖啡

silicon valley [ ] na. 硅谷 [网络] 矽谷；美国硅谷；硅谷中心地带

simple matrix [ ] 单纯矩阵

specific intent [ ] [网络] 特定意图；具体意图；具体故意

stack sequence [ ] un. 栈序列

Stanford University [ ] n. 斯坦福大学 [网络] 史丹佛大学；美国斯坦福大学；史丹福大学

steer wheel [ ] [网络] 方向盘；驾驶方向盘

text-to-speech synthesis [ ] [网络] 文本与语音合成；文字语音合成；文字至语音合成

the Creator [ ] n. 造物者 [网络] 造物主；创造者；万物的创造者

the pipeline [ ] 管道

the Slot [ ] [网络] 斯罗特浴室

to encode [ ] [网络] 编码；内码；骗码

to enroll [ ] [网络] 报名；注册；予以注册

to enroll,to enroll [ ] 《英汉医学词典》to enroll,to enroll 予以注册

to plug [ ] [网络] 插销

to roast [ ] [网络] 烤；焙烧；烤章鱼小丸子

to tag [ ] [网络] 一个远端标签

ton of [ ] 大量,许多


惯用语
and so on
and then
does that make sense
does that makes sense
for sure
i don't know
i just enrolled you
i want to enroll in cs106a for winter 2019 to learn coding
in a
you know



单词释义末尾数字为词频顺序
 zk/中考 gk/中考 ky/考研 cet4/四级 cet6/六级 ielts/雅思 toefl/托福 gre/GRE
* 词汇量测试建议用 testyourvocab.com