{"id":1477,"date":"2021-10-08T16:46:00","date_gmt":"2021-10-08T14:46:00","guid":{"rendered":"https:\/\/maxfest.dk\/port\/?p=1477"},"modified":"2021-10-11T15:31:23","modified_gmt":"2021-10-11T13:31:23","slug":"avendt-maskinelaering-08-10-2021","status":"publish","type":"post","link":"https:\/\/maxfest.dk\/port\/skiftligt-materiale\/avendt-maskinelaering-08-10-2021\/","title":{"rendered":"Avendt Maskinel\u00e6ring 08\/10 2021"},"content":{"rendered":"\n<p>Jeg blander dansk og engelsk. Jeg glemmer at rette mig selv n\u00e5r jeg lytter med nogle gange.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">How to use a loss function and an optimizer to find the best parameters<\/h2>\n\n\n\n<p>We want to update the weights until the results are sufficient. This is done in a neural network.<\/p>\n\n\n\n<p>The loss function defines how good a model performs.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">Optimizer<\/h2>\n\n\n\n<p>It is necesary to know what parameters are beeing optimized.<\/p>\n\n\n\n<p>Hvis man arbejder videre p\u00e5 en model efter at have gemt den i en ny session, s\u00e5 f\u00e5r man ikke n\u00f8dvendigvis de samme resultater som at have arbejdet videre i samme session.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Tips:<\/h3>\n\n\n\n<p>Start altid med adam optimizeren. Den er ofte best. Brug den som benchmark, og anved s\u00e5 andre.<\/p>\n\n\n\n<p>Neural network er gode til at h\u00e5ndtere multidimentionelt data.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Batch gradient descent<\/h3>\n\n\n\n<p>Batch gradient descent computes the gradient of the cost function. It uses the previous parameters as a base.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Stochastic gradient decent SGD<\/h3>\n\n\n\n<p>Create volitile path down to minimum.<\/p>\n\n\n\n<p>Can be exploitet, but can be bad.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Mini batch decent<\/h3>\n\n\n\n<p>Laver tilf\u00e6ldige valg fra data (slices).<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Momentum<\/h3>\n\n\n\n<p>Med SGD er momentum simpelt. Det kan v\u00e6re at flytte gennemsnittet, s\u00e5 det bliver mere udlignet. Det ligner gradient decent.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Nesterov accelerated gradient<\/h3>\n\n\n\n<p>Ligesom med momentum, men tager ogs\u00e5 h\u00f8jde for hele funktionen. Det er ikke \u00e5benlyst hvorfor det virker bedre, men nogen gange g\u00f8r det.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">Lave netv\u00e6rk<\/h2>\n\n\n\n<p>Man kan med keras tensorflow bruge .add for at tilf\u00f8je lag til netv\u00e6rk. Det g\u00f8r scripts nemmere at l\u00e6se.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">Ustrukturerert data<\/h2>\n\n\n\n<p>Data er ofte ustruktureret.<\/p>\n\n\n\n<p>Tids data kan nogen gange v\u00e6re kategoriseret som ustruktureret.<\/p>\n\n\n\n<p>Afh\u00e6nigt data har ulempen at det afh\u00e6nger af de initielle observationer.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">Tr\u00e6ning i for lang tid<\/h2>\n\n\n\n<p>Der er ikke en klar m\u00e5de at vide hvorn\u00e5r man skal stoppe. Men bliver man ved for l\u00e6nge bliver modellerne overfittet.<\/p>\n\n\n\n<p>Derfor kan man anvende early stopping.<\/p>\n\n\n\n<p>Siden at det foreg\u00e5r iterativt (i epochs), s\u00e5 kan man unders\u00f8ge hvorn\u00e5r en gradient n\u00e6rmer sig 0 (det bliver m\u00e5ske aldrig 0).<\/p>\n\n\n\n<p>Ellers kan man predifinere hvor mange epochs skal v\u00e6re med fra starten.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">Essentielle steps til Neural network i tensorflow<\/h2>\n\n\n\n<p>sparse_categorical_crossentropy er i nogle tilf\u00e6lde en v\u00e6rdi fra 1-9 eller noget i den stil. Jeg fik kun det halvde med pga. toilet bes\u00f8g. I andre tilf\u00e6lde tilf\u00f8jer man 0 som v\u00e6rdi.<\/p>\n\n\n\n<p>I keras tensorflow history objekter kan man finde loss, val_loss, accuracy og cal_accuracy. Det er is\u00e6r pratisk n\u00e5r man vil plotte resultater.<\/p>\n\n\n\n<p>I et tensorflow kan man f\u00f8lge udviklingen, og s\u00e5 kan man stoppe modellen n\u00e5r det passer en, i stedet for at lade den k\u00f8re x iterationer f\u00e6rdigt.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">Fundament for succesfult neural network<\/h2>\n\n\n\n<ul class=\"wp-block-list\"><li>Passende data<\/li><li>Passende arkitektur<\/li><li>God loss funktion<\/li><li>God optimizer<\/li><li>Find god balance mellem bias og varians (og undg\u00e5 overfitting)<\/li><\/ul>\n\n\n\n<h2 class=\"wp-block-heading\">Learning rate<\/h2>\n\n\n\n<p>I mange tilf\u00e6lde s\u00e5 er langsom l\u00e6ringsrate langsom til at finde minimum, medium finder den relativt hurtigt og h\u00f8j l\u00e6ringsrate finder den ofte aldrig. Det afh\u00e6nger meget st\u00e6rkt af data, s\u00e5 se det som tommelfingerregel. Man kan bruge en kombination. Momentum foretr\u00e6kker ofte en h\u00f8j learning rate. Momentum kan f\u00e5 en ud af lokal minimum.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Learning rate og batch size i kode<\/h3>\n\n\n\n<p>Learning rate defineres i optimizer &#8220;outside&#8221; med .compile<\/p>\n\n\n\n<p>Bach difineres som parameter med .fit<\/p>\n\n\n\n<p>Tradeoff mellem de 2 er s\u00e5ledes at h\u00f8j batch size tillader h\u00f8jere l\u00e6ringsrate. Tommelfingerregel. Det afh\u00e6nger yderligere af sample size. S\u00e5 batch size relativiteren skal ses som i forholdsvis h\u00f8j batch size ud fra sample size.<\/p>\n\n\n\n<p>Der er ingen regel for hvor mange epochs der skal anvendes. Hvis man er meget i tvivl kan man pr\u00f8ve med 100, og s\u00e5 stoppe den tidligt hvis det overfitter.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">Practicalities when using neural networks<\/h2>\n\n\n\n<ul class=\"wp-block-list\"><li>Saving and loading models should be done in certain frequencies. Preferby often.<\/li><li>Terminate traning when stagnating<\/li><li>When working in teams, start using early models, while others continue training<\/li><\/ul>\n\n\n\n<h2 class=\"wp-block-heading\">Tensorboard<\/h2>\n\n\n\n<p>Med tensorboard kan man f\u00f8lge udviklingen l\u00f8bende.<\/p>\n\n\n\n<p>Nogen gange kan implementeringen v\u00e6re besv\u00e6rligt. Det kan v\u00e6re restriktivt at arbejde med.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">Fordele og ulemper ved Neural networks<\/h2>\n\n\n\n<p>Fordele:<\/p>\n\n\n\n<ul class=\"wp-block-list\"><li>Kan modeleres til alt<\/li><li>Virker p\u00e5 struktureret og ustruktureret data<\/li><li>Kan opn\u00e5 SOTA for en lang r\u00e6kke af problemer is\u00e6r n\u00e5r der er meget data og det er ustruktoreret<\/li><\/ul>\n\n\n\n<p>Ulemper:<\/p>\n\n\n\n<ul class=\"wp-block-list\"><li>Trods for at de kan bruges til alt, s\u00e5 kan det v\u00e6re n\u00e6rmest umuligt at opn\u00e5.<\/li><li>De kan v\u00e6re sv\u00e6re at forst\u00e5 (black box)<\/li><li>De er meget tilb\u00f8jelige til overfitting<\/li><li>De kan v\u00e6re besv\u00e6relige at anvende<\/li><\/ul>\n\n\n\n<p><\/p>\n","protected":false},"excerpt":{"rendered":"<p>Jeg blander dansk og engelsk. Jeg glemmer at rette mig selv n\u00e5r jeg lytter med nogle gange. How to use a loss function and an optimizer to find the best parameters We want to update the weights until the results are sufficient. This is done in a neural network. The loss function defines how good&#8230;<\/p>\n","protected":false},"author":1,"featured_media":715,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[11,10],"tags":[34,30],"class_list":["post-1477","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-noter","category-skiftligt-materiale","tag-anvendt-maskinlaering-ds807","tag-sdu"],"_links":{"self":[{"href":"https:\/\/maxfest.dk\/port\/wp-json\/wp\/v2\/posts\/1477","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/maxfest.dk\/port\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/maxfest.dk\/port\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/maxfest.dk\/port\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/maxfest.dk\/port\/wp-json\/wp\/v2\/comments?post=1477"}],"version-history":[{"count":4,"href":"https:\/\/maxfest.dk\/port\/wp-json\/wp\/v2\/posts\/1477\/revisions"}],"predecessor-version":[{"id":1488,"href":"https:\/\/maxfest.dk\/port\/wp-json\/wp\/v2\/posts\/1477\/revisions\/1488"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/maxfest.dk\/port\/wp-json\/wp\/v2\/media\/715"}],"wp:attachment":[{"href":"https:\/\/maxfest.dk\/port\/wp-json\/wp\/v2\/media?parent=1477"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/maxfest.dk\/port\/wp-json\/wp\/v2\/categories?post=1477"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/maxfest.dk\/port\/wp-json\/wp\/v2\/tags?post=1477"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}